Apache + Python

• Always mod_wsgi and turn on the daemon mode (default is embedded mode). This way multiple dedicated processes are started and reused to handle the requests. reading http://blog.dscpl.com.au/2012/10/why-are-you-using-embedded-mode-of.html by author of mod_wsgi.

Web Scraping

• I used requestjs + cheeriojs +selectorGadget and they worked pretty well for me. The same stack exists for many other languages such as Python. Python's mechanize worked great as well in the past.
• some code I used.
• scraping Yahoo finance

Front-end

• Bootstrap

Build a webapp with Flask

• ORM: Flask-SQLAlchemy. It offers some convenience functions

  • class Role(db.Model):
        id = db.Column(db.Integer(), primary_key=True)
        name = db.Column(db.String(50), unique=True)
    
        def __str__(self):
            return self.name
    
    class UserRoles(db.Model):
        id = db.Column(db.Integer(), primary_key=True)
        user_id = db.Column(db.Integer(), db.ForeignKey('user.id', ondelete='CASCADE'))
        role_id = db.Column(db.Integer(), db.ForeignKey('role.id', ondelete='CASCADE'))
    
    class User(db.Model, UserMixin):
        id = db.Column(db.Integer, primary_key=True)
    
        # User Authentication information
        username = db.Column(db.String(50), nullable=False, unique=True)
        password = db.Column(db.String(255), nullable=False, default='')
        reset_password_token = db.Column(db.String(100), nullable=False, default='')
    
        # User Email information
        email = db.Column(db.String(255), nullable=False, unique=True)
        confirmed_at = db.Column(db.DateTime())
    
        # User information
        is_enabled = db.Column(db.Boolean(), nullable=False, default=False)
        first_name = db.Column(db.String(50), nullable=False, default='')
        last_name = db.Column(db.String(50), nullable=False, default='')
    
        roles = db.relationship('Role', secondary='user_roles',
                backref=db.backref('users', lazy='dynamic'))
    
        def is_active(self):
            return self.is_enabled
    
        def is_admin(self):
            return self.is_authenticated() and self.admin == 1
    
        def __str__(self):
            return self.username +'-' +self.first_name+' '+self.last_name
        

• Database migration: Flask-Migrate
  • db.reflect()

        db.reflect()
        db.drop_all()
        

• Form: Flask-WTF
• User management: Flask-User. Flask-User embeds Flask-Login for session management.

  •     from flask.ext.login import LoginManager, login_user, logout_user, \
             login_required, current_user
    
        @app.route('/')
        @login_required
        def index():
          return render_template('index.html', user=current_user)
    
        @app.route('/')
        @roles_required('admin')  
        def admin():
          return render_template('admin.html')
        

  • One thing is that it handles everything for user signup/login including the view and controller. There are ways to customize them however: https://pythonhosted.org/Flask-User/customization.html
  • For Registrat Form: user_manager = UserManager(db_adapter, app, register_form=MyRegisterForm). Similar for view functions.
  • Post User creation: sometimes there are more work then merely creating a user in the database. e.g. create a payment profile. In this case we can subscribe to its signals: https://pythonhosted.org/Flask-User/signals.html
• Sending emails: Flask-Mail
• Unit test: Flask-Testing.
• Admin panel: Flask-Admin. Then you have functions like current_user.has_roles('admin'), @roles_required('merchant'), etc
• Modular applications: Flask Blueprint. Making it easier to organize if you have many models/views
• i18n: Flask-Babel
• Tasks: Flask-Scripts
• Frontend: Bootstrap
• Data Persistence: SQLite/MySQL/Postgre up to you. If postgres you need psycopg2
• Virtual Env
• Dependency management: pip
• Task management: Fabric