The use of Open Source Routing Machine is not always that straightforward and simple as it may seem at first. Very often additional factors should be considered. Server choice and its proper configuration are definitely one of them and we learned it from our own experience.
We used to have a project to develop a logistic service for goods transportation. It was in Ruby and the main algorithm was Ruin And Recreate. We also used Amazon instance. During the development process we had to calculate transportation time. The best option was to use OSRM. So we created a gem to support OSRM API.
OSRM project had quite good documentation to build the project.
git clone https://github.com/Project-OSRM/osrm-backend.git cd osrm-backend mkdir -p build cd build cmake .. make sudo make install
The routing engine was supposed to include New York and its suburbs. To get a list of maps we used geofabrik.de.
We started working on OSRM location map and used New York maps for this (about 650 MB).
It was t2.medium instance used so we ran the first command with no much analysis.
./osrm-extract us-northeast-latest.osm.pbf
Shortly there appeared the following line:
[warn] Exception occured: std::bad_alloc
Having consulted the documentation we noticed that OSRM utilities were very memory consuming. So we configured a .stxxl file to use external memory.
(nano) disk=./STXXL,2000,syscall
However, we got the same error again. Therefore, it was necessary to increase the swapfile to 4GB and we had to set capacity attribute to 4000 in the .stxxl file.
Then we proceeded with map generation.
./osrm-prepare us-northeast-latest.osrm ./osrm-routed us-northeast-latest.osrm
Operating storage and processor were 100% loaded during map generation. It took about 70-80 minutes for our utilities to process New York maps (instance t2.medium).
At that stage we could get data required for our logistic project.
Later we needed to add Washington and its suburbs as a new location.
So again we used geofabrik.de and got necessary data. We received us-south-latest.pbf file (1.7 GB).
Then we had to join two locations - us-northeast and us-south. To do it we used Osmconvert utility. There were no problems with the installation.
Afterwards we united the locations running the following command.
osmconvert us-northeast-latest.osm.pbf us-south-latest.osm.pbf -o=us-east-coast.osm.pbf
The following step was to run nohup to process data.
Next day it appeared that for 24 hours the utility had loaded Amazon instance by 100% (CPU and RAM).
Moreover, the server stopped SSH connection. After server reboot we decided to upgrade instance to something more powerful. As a result, we cloned the current instance and used it on other configuration to get better results – c4.4xlarge.
Surprsingly, it took 27 minutes for the new configuration to process data. It was really fast. Htop utility showed the following results: CPU – 1400%, RAM – 20-25 %.
Conclusion
Server productivity and configuration are very important when you use OSRM in cartographic service. The higher, the better.