request-quote
Ruby on Rails

You Shall Not Pass Unless You Know How To Build Routes

By Igor P. January 19th, 2016

The use of Open Source Routing Machine is not always that straightforward and simple as it may seem at first. Very often additional factors should be considered. Server choice and its proper configuration are definitely one of them and we learned it from our own experience.  

We used to have a project to develop a logistic service for goods transportation. It was in Ruby and the main algorithm was Ruin And Recreate. We also used Amazon instance. During the development process we had to calculate transportation time. The best option was to use OSRM. So we created a gem to support OSRM API.

OSRM project had quite good documentation to build the project.

 

git clone https://github.com/Project-OSRM/osrm-backend.git
cd osrm-backend
mkdir -p build
cd build
cmake ..
make
sudo make install

 

The routing engine was supposed to include New York and its suburbs. To get a list of maps we used geofabrik.de.

We started working on OSRM location map and used New York maps for this (about 650 MB). 

It was t2.medium instance used so we ran the first command with no much analysis. 

 

./osrm-extract us-northeast-latest.osm.pbf

 

Shortly there appeared the following line:

 

[warn] Exception occured: std::bad_alloc

 

Having consulted the documentation we noticed that OSRM utilities were very memory consuming. So we configured a .stxxl file to use external memory. 

 

(nano)
disk=./STXXL,2000,syscall

 

However, we got the same error again. Therefore, it was necessary to increase the swapfile to 4GB and we had to set capacity attribute to 4000 in the .stxxl file. 

Then we proceeded with map generation. 

 

./osrm-prepare us-northeast-latest.osrm
./osrm-routed us-northeast-latest.osrm

 

Operating storage and processor were 100% loaded during map generation. It took about 70-80 minutes for our utilities to process New York maps (instance t2.medium).

At that stage we could get data required for our logistic project.

Later we needed to add Washington and its suburbs as a new location.

So again we used  geofabrik.de and got necessary data. We received us-south-latest.pbf file (1.7 GB).

Then we had to join two locations - us-northeast and us-south. To do it we used Osmconvert utility. There were no problems with the installation. 

Afterwards we united the locations running the following command. 

 

osmconvert us-northeast-latest.osm.pbf us-south-latest.osm.pbf -o=us-east-coast.osm.pbf

 

The following step was to run nohup to process data.

Next day it appeared that for 24 hours the utility had loaded Amazon instance by 100% (CPU and RAM).

Moreover, the server stopped SSH connection. After server reboot we decided to upgrade instance to something more powerful. As a result, we cloned the current instance and used it on other configuration to get better results – c4.4xlarge. 

Surprsingly, it took 27 minutes for the new configuration to process data. It was really fast. Htop utility showed the following results: CPU – 1400%, RAM – 20-25 %. 

 

Conclusion

Server productivity and configuration are very important when you use OSRM in cartographic service. The higher, the better. 

 

Igor P.

Igor P.

Ruby on Rails Developer at iKantam

Browse Recent Posts