Setup CartoDB dev environment on Mac OS X

“CartoDB is a geospatial database on the cloud that allows for the storage and visualization of data on the web. Using CartoDB will allow you to quickly create map based visualizations.” (cartodb.com)

This document walks you step by step to have a development CartoDB running on your Mac OS X

Step 1 – Prerequisites

  • First of all, install Ruby if you haven’t (recommend using RVM or rbenv)
  • Install homebrew if you haven’t
    ruby -e "$(curl -fsSL https://raw.github.com/mxcl/homebrew/go/install)"
  • Or, update homebrew if you already have it
    brew update

Step 2 – CartoDB Dependencies

  • brew install geos
  • brew install gdal --with-postgres --with-postgresql

    When complete, follow brew instructions to initialize a database, start postgresql and have launchd to start postgresql at login if you like.

  • brew install json-c
  • brew install postgis
  • Create a bash script with following lines and execute it
    #!/usr/bin/env bash
    POSTGIS_SQL_PATH=/usr/local/share/postgis/
    
    createdb -E UTF8 template_postgis
    
    createlang -d template_postgis plpgsql
    
    psql -d postgres -c \
    
    "UPDATE pg_database SET datistemplate='true' WHERE datname='template_postgis'"
    
    psql -d template_postgis -f $POSTGIS_SQL_PATH/postgis.sql
    
    psql -d template_postgis -f $POSTGIS_SQL_PATH/spatial_ref_sys.sql
    
    psql -d template_postgis -f $POSTGIS_SQL_PATH/legacy.sql
    
    psql -d template_postgis -f $POSTGIS_SQL_PATH/rtpostgis.sql
    
    psql -d template_postgis -f $POSTGIS_SQL_PATH/topology.sql
    
    psql -d template_postgis -c "GRANT ALL ON geometry_columns TO PUBLIC;"
    
    psql -d template_postgis -c "GRANT ALL ON spatial_ref_sys TO PUBLIC;"
  • brew install node
  • brew install redis

    When complete, start up redis-server

  • sudo easy_install pip
  • Clone cartodb codebase to your machine
    git clone --recursive https://github.com/CartoDB/cartodb.git
    cd cartodb
    sudo pip install -r python_requirements.txt
  • By the time of writing this doc, CartoDB doesn’t work with latest stable version of varnish, so a bit extra work to install varnish v2.1.5
    cd `brew --prefix`
    
    git checkout 3d52e52 Library/Formula/varnish.rb #varnish v2.1.5
    
    brew install varnish
  • brew install mapnik --with-gdal --with-geos --with-postgresql
  • And for the sake of unzipping .zip file
    brew install unp

Step 3 – Setup CartoDB components

A. CartoDB SQL API

  • git clone git://github.com/CartoDB/CartoDB-SQL-API.git
    
    cd CartoDB-SQL-API
    
    git checkout master
  • Edit npm-shrinkwrap.json to change zipfile version from 0.3.4 to 0.4.2, then
    npm install -g node-gyp
    
    npm install
    
    cp config/environments/development.js.example config/environments/development.js
    
    node app.js development

    You should not see any error messages. Follow the rest of the instructions in a new tab (or terminate the running app using Ctrl+c).

B. Windshaft-cartodb

  • git clone git://github.com/CartoDB/Windshaft-cartodb.git
    
    cd Windshaft-cartodb
    
    git checkout master
  • Edit npm-shrinkwrap.json to change mapnik version from 0.7.22 to 0.7.25, then
    npm install
    
    cp config/environments/development.js.example config/environments/development.js
    
  • Edit config/environments/development.js line #31 change undefined to ‘2.1.1’
    node app.js development

    You should not see any error messages. Follow the rest of the instructions in a new tab (or terminate the running app using Ctrl+c).

C. cartodb (CartoDB Dashboard)

  • Go back to your previous checked out cartodb repo
    cd ../cartodb
    
    bundle install
    
    cp config/app_config.yml.sample config/app_config.yml
    
    cp config/database.yml.sample config/database.yml
  • Add postgres role in your local postgresql and grant admin priviledges, then update database.yml accordingly
  • export SUBDOMAIN=development
    
    export EMAIL=user@example.com
    
    export PASSWORD=111111
    
    bundle exec rake cartodb:db:setup
    
    echo "127.0.0.1 ${SUBDOMAIN}.localhost.lan" | sudo tee -a /etc/hosts

    You are free to change the domain name as you like. But if you do, make sure you update config/app_config.yml accordingly. I recommend you leave it as it is.

  • Make sure redis is running, then
    ./script/restore_redis
    
  • Start up a resque worker to process uploaded files
    QUEUE=* bundle exec rake resque:work

    Again, you should not see any error messages. Follow the rest of the instructions in a new tab (or terminate the running app using Ctrl+c).

  • bundle exec rails s

If everything is going well (and with a bit of luck), you should have a running CartoDB at http://development.localhost.lan:3000/

You can log in with the username/password you specified earlier, e.g development/111111. (Yes, subdomain is your username)

D. Possible Issues

  • If you have an issue with ActionView::Template::Error (couldn't find file 'cdb/vendor/jquery.min'..., maybe you haven’t executed checkout cartodb recursively, do this
    git submodule update --init
  • If you come across [TypeError: Arguments to path.resolve must be strings], comment out line #72 of Windshaft-cartodb/node_modules/windshaft/node_modules/tilelive-mapnik/lib/mapnik_backend.js, and restart Windshaft-cartodb
    // uri.pathname = path.resolve(uri.pathname);

E. Tip

After testing all components work smoothly, you can use foreman instead of starting cartodb, windsharft and sql api separately

bundle exec foreman start -p 3000

 

All done & have fun!

 


 

Advanced steps start from here (follow instructions beyond this point only if you want to run CartoBD in production mode locally)

 

To launch varnishd

  • Edit /usr/local/etc/varnish/default.vcl and uncomment line #7~10
    sudo varnishd -f /usr/local/etc/varnish/default.vcl -s malloc,1G -T 0.0.0.0:6082 -a 0.0.0.0:6081

Some config tweeks for cartodb

  • In app/controllers/application_controller.rb, comment out line #18 & #25 to disable SSL
  • In config/environments/production.rb, change server_static_assets to true, then
    bundle exec rake assets:precompile
  • In config/app_config.yml, add the following configration under production
    tiler_domain: 'localhost.lan'
    
    tiler_port: 8181
    
    tiler_protocal: 'http'

For Windshaft-cartodb

  • Take the example prod config
    cp config/environments/production.js.example config/environments/production.js
  • edit config/environments/production.js line #19 to change the postgres port to 5432 or whatever your postgresql is using
  • edit config/environments/production.js line #25 to change the mapnik_version to ‘2.1.1’
  • edit config/environments/production.js line #34 to change the cache_basedir to ‘/tmp/tile_assets/’
  • edit config/environments/production.js line #51 to change protocal to ‘http’, version to ‘v1’

For CartoDB-SQL-API

  • Take the example prod config
    cp config/environments/production.js.example config/environments/production.js
  • edit config/environments/production.js line #10 to change db_port to ‘5432’ or whatever your postgresql is using

Create user account in prod db

  • export SUBDOMAIN=production
    
    export EMAIL=user@example.com
    
    export PASSWORD=111111
    
    RAILS_ENV=production r cartodb:db:setup
    
    bundle exec rake assets:precompile
    
    echo "127.0.0.1 ${SUBDOMAIN}.localhost.lan" | sudo tee -a /etc/hosts

Edit Procfile to use production mode instead of development mode, then

  • bundle exec foreman start -p 3000
Now you should be able to have all components running in production mode.