Setup CartoDB dev environment on Mac OS X

“CartoDB is a geospatial database on the cloud that allows for the storage and visualization of data on the web. Using CartoDB will allow you to quickly create map based visualizations.” (cartodb.com)

This document walks you step by step to have a development CartoDB running on your Mac OS X

Step 1 – Prerequisites

  • First of all, install Ruby if you haven’t (recommend using RVM or rbenv)
  • Install homebrew if you haven’t
    ruby -e "$(curl -fsSL https://raw.github.com/mxcl/homebrew/go/install)"
  • Or, update homebrew if you already have it
    brew update

Step 2 – CartoDB Dependencies

  • brew install geos
  • brew install gdal --with-postgres --with-postgresql

    When complete, follow brew instructions to initialize a database, start postgresql and have launchd to start postgresql at login if you like.

  • brew install json-c
  • brew install postgis
  • Create a bash script with following lines and execute it
    #!/usr/bin/env bash
    POSTGIS_SQL_PATH=/usr/local/share/postgis/
    
    createdb -E UTF8 template_postgis
    
    createlang -d template_postgis plpgsql
    
    psql -d postgres -c \
    
    "UPDATE pg_database SET datistemplate='true' WHERE datname='template_postgis'"
    
    psql -d template_postgis -f $POSTGIS_SQL_PATH/postgis.sql
    
    psql -d template_postgis -f $POSTGIS_SQL_PATH/spatial_ref_sys.sql
    
    psql -d template_postgis -f $POSTGIS_SQL_PATH/legacy.sql
    
    psql -d template_postgis -f $POSTGIS_SQL_PATH/rtpostgis.sql
    
    psql -d template_postgis -f $POSTGIS_SQL_PATH/topology.sql
    
    psql -d template_postgis -c "GRANT ALL ON geometry_columns TO PUBLIC;"
    
    psql -d template_postgis -c "GRANT ALL ON spatial_ref_sys TO PUBLIC;"
  • brew install node
  • brew install redis

    When complete, start up redis-server

  • sudo easy_install pip
  • Clone cartodb codebase to your machine
    git clone --recursive https://github.com/CartoDB/cartodb.git
    cd cartodb
    sudo pip install -r python_requirements.txt
  • By the time of writing this doc, CartoDB doesn’t work with latest stable version of varnish, so a bit extra work to install varnish v2.1.5
    cd `brew --prefix`
    
    git checkout 3d52e52 Library/Formula/varnish.rb #varnish v2.1.5
    
    brew install varnish
  • brew install mapnik --with-gdal --with-geos --with-postgresql
  • And for the sake of unzipping .zip file
    brew install unp

Step 3 – Setup CartoDB components

A. CartoDB SQL API

  • git clone git://github.com/CartoDB/CartoDB-SQL-API.git
    
    cd CartoDB-SQL-API
    
    git checkout master
  • Edit npm-shrinkwrap.json to change zipfile version from 0.3.4 to 0.4.2, then
    npm install -g node-gyp
    
    npm install
    
    cp config/environments/development.js.example config/environments/development.js
    
    node app.js development

    You should not see any error messages. Follow the rest of the instructions in a new tab (or terminate the running app using Ctrl+c).

B. Windshaft-cartodb

  • git clone git://github.com/CartoDB/Windshaft-cartodb.git
    
    cd Windshaft-cartodb
    
    git checkout master
  • Edit npm-shrinkwrap.json to change mapnik version from 0.7.22 to 0.7.25, then
    npm install
    
    cp config/environments/development.js.example config/environments/development.js
    
  • Edit config/environments/development.js line #31 change undefined to ‘2.1.1’
    node app.js development

    You should not see any error messages. Follow the rest of the instructions in a new tab (or terminate the running app using Ctrl+c).

C. cartodb (CartoDB Dashboard)

  • Go back to your previous checked out cartodb repo
    cd ../cartodb
    
    bundle install
    
    cp config/app_config.yml.sample config/app_config.yml
    
    cp config/database.yml.sample config/database.yml
  • Add postgres role in your local postgresql and grant admin priviledges, then update database.yml accordingly
  • export SUBDOMAIN=development
    
    export EMAIL=user@example.com
    
    export PASSWORD=111111
    
    bundle exec rake cartodb:db:setup
    
    echo "127.0.0.1 ${SUBDOMAIN}.localhost.lan" | sudo tee -a /etc/hosts

    You are free to change the domain name as you like. But if you do, make sure you update config/app_config.yml accordingly. I recommend you leave it as it is.

  • Make sure redis is running, then
    ./script/restore_redis
    
  • Start up a resque worker to process uploaded files
    QUEUE=* bundle exec rake resque:work

    Again, you should not see any error messages. Follow the rest of the instructions in a new tab (or terminate the running app using Ctrl+c).

  • bundle exec rails s

If everything is going well (and with a bit of luck), you should have a running CartoDB at http://development.localhost.lan:3000/

You can log in with the username/password you specified earlier, e.g development/111111. (Yes, subdomain is your username)

D. Possible Issues

  • If you have an issue with ActionView::Template::Error (couldn't find file 'cdb/vendor/jquery.min'..., maybe you haven’t executed checkout cartodb recursively, do this
    git submodule update --init
  • If you come across [TypeError: Arguments to path.resolve must be strings], comment out line #72 of Windshaft-cartodb/node_modules/windshaft/node_modules/tilelive-mapnik/lib/mapnik_backend.js, and restart Windshaft-cartodb
    // uri.pathname = path.resolve(uri.pathname);

E. Tip

After testing all components work smoothly, you can use foreman instead of starting cartodb, windsharft and sql api separately

bundle exec foreman start -p 3000

 

All done & have fun!

 


 

Advanced steps start from here (follow instructions beyond this point only if you want to run CartoBD in production mode locally)

 

To launch varnishd

  • Edit /usr/local/etc/varnish/default.vcl and uncomment line #7~10
    sudo varnishd -f /usr/local/etc/varnish/default.vcl -s malloc,1G -T 0.0.0.0:6082 -a 0.0.0.0:6081

Some config tweeks for cartodb

  • In app/controllers/application_controller.rb, comment out line #18 & #25 to disable SSL
  • In config/environments/production.rb, change server_static_assets to true, then
    bundle exec rake assets:precompile
  • In config/app_config.yml, add the following configration under production
    tiler_domain: 'localhost.lan'
    
    tiler_port: 8181
    
    tiler_protocal: 'http'

For Windshaft-cartodb

  • Take the example prod config
    cp config/environments/production.js.example config/environments/production.js
  • edit config/environments/production.js line #19 to change the postgres port to 5432 or whatever your postgresql is using
  • edit config/environments/production.js line #25 to change the mapnik_version to ‘2.1.1’
  • edit config/environments/production.js line #34 to change the cache_basedir to ‘/tmp/tile_assets/’
  • edit config/environments/production.js line #51 to change protocal to ‘http’, version to ‘v1’

For CartoDB-SQL-API

  • Take the example prod config
    cp config/environments/production.js.example config/environments/production.js
  • edit config/environments/production.js line #10 to change db_port to ‘5432’ or whatever your postgresql is using

Create user account in prod db

  • export SUBDOMAIN=production
    
    export EMAIL=user@example.com
    
    export PASSWORD=111111
    
    RAILS_ENV=production r cartodb:db:setup
    
    bundle exec rake assets:precompile
    
    echo "127.0.0.1 ${SUBDOMAIN}.localhost.lan" | sudo tee -a /etc/hosts

Edit Procfile to use production mode instead of development mode, then

  • bundle exec foreman start -p 3000
Now you should be able to have all components running in production mode.
  • Mancuso74

    Hi!! What is the correct version of ruby that must be installed?

    • Leon Xiang

      Hi Mancuso74, I’m sorry for the late response. The comment I posted a couple days ago got lost for some reason 🙁

      CartoDB requires Ruby 1.9.2. So any patch version of Ruby 1.9.3 and 2.0 should do. I’m using 1.9.3-p448 when writing this post.

      • Mancuso74

        Thanks!! I would like also to know what to do here
        “Add postgres role in your local postgresql and grant admin priviledges, then update database.yml accordingly”

        • Leon Xiang

          Right, I didn’t put down the details here as it’s not really related to CartoDB.

          What you need to do here is have a admin role created for your local postgresql (so it can create/drop databases) and tell rails to connect to the database with that credential. You can achieve this either through a postgresql client such as navicat or command line.

          Here is what you can do:
          # Connect to local postgres
          psql

          # Add postgres user if it doesn’t exist
          CREATE USER postgres WITH PASSWORD ‘myPassword’;

          # Grant admin privileges to this user
          GRANT ALL PRIVILEGES ON ALL TABLES IN SCHEMA public TO postgres;

          # Exit
          q

          # Test the user you’ve just created
          psql -Upostgres

          # If that works, edit cartodb config/database.yml with:
          username: postgres
          password: myPassword

          Please reference to PostgreSQL and Rails documentation for more details about this.

          • Mancuso74

            thanks a lot for your quick and precise answer. I know it is not directly related to CartoDB but i know that a lot of people not experienced in unix, postgres, ruby and other things are interested in testing and using Cartodb and this kind of help would be very ppreciated 🙂 Thanks again.

            And now, the last question: When i do ‘bundle install’ in my cartodb directory, i get an error while installing therubyracer. I forced the GEMFILE to install therubyracer 0.12.0 instead of 0.9.10… my question is? will this version bring problems during the rest of the install ? My actual version of ruby is 1.9.2 If I try with 2.0.0 or 2.1.0 i have other errors (while installing gem debugger-linecache). While if i change version in GEMFILE, the budge install command ends without any errors…

            Thanks a lot again!!!!

          • Mancuso74

            Of course, i have to thank you for this great ‘howto’ and tell you that i am about to get it running and that i couldn’t do it without your indications!

          • Leon Xiang

            No worries. Glad to help.

            As for myself, therubyracer 0.9.10 bundled without any problem. However I heard one of my coworkers had the same issue when bundle installing therubyracer 0.9.10 and removing the locked version in Gemfile (in which case v0.12.0 is installed instead) fixed the problem for him. I wouldn’t worry about the version too much.

            Regarding to the ruby version issue, my guess is the debugger-linecache gem is not compatible with ruby 2.0.

            Thanks for posting the issues you came across here, I’m sure it will be beneficial to others.

  • Tan Le

    I need to install numpy first before install gdal

    $ pip install numpy

  • Tan Le

    Running

    brew install gdal --with-postgres --with-postgresql

    on Mac OS 10.9 Mavericks throws:

    Error: gdal dependency geos was built with the following
    C++ standard library: libc++ (from clang)
    This is incompatible with the standard library being used
    to build gdal: libstdc++ (from clang)

    Throwing an extra arg seems to do the trick

    brew install gdal --env=std --with-postgres --with-postgresql

  • Tan Le

    Few notes on installing Windshaft-cartodb steps:

    1. Make sure you set the right version of mapnik in development.js (the latest via homebrew is 2.2.0)

    brew info mapnik | head -n1

    2. If you are on Mavericks 10.9 and seeing the following error when running node app.js development

    dyld: lazy symbol binding failed: Symbol not found: __ZN6mapnik15freetype_engine14register_fontsERKSsb
    Referenced from: /Users/tan_le/Sandbox/Windshaft-cartodb/node_modules/mapnik/lib/_mapnik.node
    Expected in: dynamic lookup

    dyld: Symbol not found: __ZN6mapnik15freetype_engine14register_fontsERKSsb
    Referenced from: /Users/tan_le/Sandbox/Windshaft-cartodb/node_modules/mapnik/lib/_mapnik.node
    Expected in: dynamic lookup

    Trace/BPT trap: 5

    The workaround is:

    rm -rf node_modules/
    export CXXFLAGS="-mmacosx-version-min=10.9"
    export CFLAGS="-mmacosx-version-min=10.9"
    export LDFLAGS="-mmacosx-version-min=10.9"
    npm install

  • Pingback: Todos los avistamientos de OVNIS en un mapa interactivo con CartoDB - Protips y más()

  • Alex C-W

    Hi Tan,

    Thanks for putting these instructions online.

    We’re trying to install given your instructions above. However, on the bash script step, we keep getting a bunch of errors like the following:
    psql:/usr/local/share/postgis/postgis.sql:93: ERROR: could not load library “/usr/local/Cellar/postgresql/9.3.3/lib/postgis-2.1.so”: dlopen(/usr/local/Cellar/postgresql/9.3.3/lib/postgis-2.1.so, 10): Library not loaded: libxml2.2.dylib
    Referenced from: /usr/local/Cellar/postgresql/9.3.3/lib/postgis-2.1.so
    Reason: Incompatible library version: postgis-2.1.so requires version 12.0.0 or later, but libxml2.2.dylib provides version 10.0.0
    psql:/usr/local/share/postgis/postgis.sql:98: ERROR: current transaction is aborted, commands ignored until end of transaction block
    psql:/usr/local/share/postgis/postgis.sql:105: ERROR: current transaction is aborted, commands ignored until end of transaction block
    psql:/usr/local/share/postgis/postgis.sql:113: ERROR: current transaction is aborted, commands ignored until end of transaction block
    psql:/usr/local/share/postgis/postgis.sql:118: ERROR: current transaction is aborted, commands ignored until end of transaction block

    Did you have similar problems? Might you have any clues on how to fix this?

    • Tan Le

      Hi Alex,

      A little bit of a late response. I haven’t seen that error before.

      However, it seems like you have a incompatible version of libxml2 installed. Did you have libxml2 installed and symlinked via homebrew?


      brew info libxml2

      • Alex C-W

        Thanks for the reply anytime, Tan! I’ll take a look soon and try that out.