Open source navigation with OpenTripPlanner

3 minute read

Published:

OpenTripPlanner-logo

OpenTripPlanner (OTP) is an open source multi-modal (i.e. walk/car/bicycle/transit/..) trip planner engine written in Java with an easy-to-use executable. It has amazing capabilities in terms of consuming the data from OpenStreetMap (OSM), General Transit Feed Specification (GTFS) as well as General Bikeshare Feed Specification (GBFS) feed. It comes with quite good documentation and a continously growing community.

Getting started

For me, the easiest way to start experimenting is with docker, thus, avoiding the headache to install/compile dependent libraries. I forked the a docker-otp image for OTP and added few more documentation to it and a newer version is available on my github which needs to build and is also adjusted for newer version of the OTP. After cloning the repo, one can quickly build the graphs as per the documentation. To build a graph, one needs the OSM data, which can be downloaded from geofabrik and for a small memory machine, it is a good idea to crop the region to the ROI (maybe a city). Instruction to do so can be found in a previous post. Additionally, one would need the GTFS feed, if you want to use the public transport data. For some big cities, this kind of data is opensourced and a zip file can be easily downloaded. Please note, if either on of data is not available then it will affect only that transport mode (OSM for bicycle/walk/car and GTFS for public transport), thereby, we can build the graph with the limited data only. Reiterating from the documentation, we need to place OSM and GTFS data to docker-otp/graphs/city_name, then to build the graph

docker run -v $PWD:/graphs -e JAVA_OPTIONS=-Xmx4G your_docker_id/otp --build /graphs

After building the graph, we can easily turn on the server:

docker run -p 8080:8080 -v $PWD:/var/otp/graphs -e JAVA_OPTIONS=-Xmx4G otp --server --autoScan --verbose

This will start a Java based navigation service and can be viewed with localhost:8080 in the browser. This service would also enable us to query the data via its REST API. E.g.

http://localhost:8080/otp/routers/default/plan?fromPlace=28.658420, 77.230757&toPlace=28.544442, 77.206334&mode=TRANSIT,WALK

However, the response would be xml. To have a json response, we need to pass the header, for e.g., in python, we can pass the following header in our request: headers={"Accept-type": "application/json"} With this basic setup, one can build a service or an application. Checkout this notebook for a ready-to-use solution available in rasta for OTP.

Taking a step

Adding bike rental

With the smart mobility, many cities are promoting bike rentals and scooters for the last mile connectivity. This could be integrated within the OTP quity easily. To use it without any coding in the source code, we need to have the access to the real-time GBFS feed (real-time because bike availability will be keep changing at all stations). A list of availavle GBFS feed: https://github.com/NABSA/gbfs/blob/master/systems.csv

After having access to the right GBFS feed for the city, you need to add this in the configuration file (named router-config.json) of that city (docker-otp/graphs/name_of_city folder) and it’s good idea to rebuild the graph. Following is a sample router-config.json file:

// router-config.json
{
    // Routing defaults are any public field or setter in the Java class
    // org.opentripplanner.routing.core.RoutingRequest
    "routingDefaults": {
        "numItineraries": 6,
        "walkSpeed": 2.0,
        "stairsReluctance": 4.0,
        "carDropoffTime": 240
    },

    {
        "requestLogFile": "/var/otp/graphs/request.log"
    }

    "updaters": [

        // Bike parking availability
        {
            "type": "bike-park"
        },

        {
             "type": "bike-rental",
             "frequencySec": 60,
             "sourceType": "gbfs",
             "url": "https://gbfs.nextbike.net/maps/gbfs/v1/nextbike_dd/de/"
        }

    ]
}

This file will overwrite the default settings. For a detailed list of the keywords, refer to the official document.

I will try to document a more deep dive by updating this post in future.

You May Also Enjoy

Setting up an Overpass API server with Docker

3 minute read

Published:

You might have already seen my blog on OpenStreetMap from 2020. In that post, I briefly talked about Overpass API server with a pointer to a GitHub repo to setting up locally. However, that setup might fail if we try to build for entire planet. Additionally, source repository hasn’t been updated for 5 years. I’ve received some requests to assist in setting up the Overpass server. Therefore, this short post basically illustrates the process of local Overpass server.

Starting your own Overpass API server

Currently, the process has been very simple and straightforward, thanks to this docker image docker image.

Building the server from raw OSM files

This approach is beneficial when our region of interest in small (compared to entire world).

docker run -e OVERPASS_META=yes -e OVERPASS_MODE=init -e OVERPASS_PLANET_URL=http://download.geofabrik.de/europe/monaco-latest.osm.bz2 -e OVERPASS_DIFF_URL=http://download.openstreetmap.fr/replication/europe/monaco/minute/ -e OVERPASS_RULES_LOAD=10 -v /overpass_db/:/db/overpass_clone_db -p 8888:80 -it --name overpass_monaco wiktorn/overpass-api

This usually takes 5 minutes on a normal computer including the downloading image from dockerhub, downloading OSM file from geofabrik and building the database for a 700 KB file. All the generated builds can be used with the server. At the end of this build, docker container will be stopped and need to be started again. For the same, either one can assign a name to container in the previous step or can look at auto-generated name with docker ps --all command. After grabbing the name, simply start the container as docker start <CONTAINER NAME>. Alternatively, one can pass -e OVERPASS_STOP_AFTER_INIT false option so that we can continue the instance after flushing database.

img

With that, we can query for a pizza shop: -.

In the above, we didn’t use a custom region as shown here obtained from OSM tool without tweaks. But, workaround should be simple, i.e. either providing file as file:/// as mention here or hosting file with local HTTP server.

Note: So far, I am not able to circumvent the issue when I try to mount a local folder in current directory. Although build was successful however during the query server results in error. This is specific to windows only.

Cloning for entire world

When we need to scale up to entire world, then cloning is a better option compared to building from the raw OSM file. In this case, we can pass the option in OVERPASS_MODE for clone. The data will be cloned from the Overpass API server with the defined replication. You can check out available replication frequency on the OSM wiki.

docker run -e OVERPASS_MODE=clone -e OVERPASS_DIFF_URL=https://planet.openstreetmap.org/replication/day/ -v /big/docker/overpass_clone_db/:/db -p 8888:80 -it --name overpass_world wiktorn/overpass-api

This process took approx 2 hours with a good internet speed on a Linux machine, and it took approx 204 GB. For sure, this number will grow with more contributions. We can now query for nearby Indian restaurants from our POI with query: -

Using in Python

I have a sample REST API with Flask which basically finds the nearest toilets from the query point. To use the above server is easy, we just need to change the URL of Overpass API server mentioned here to self.sever = 'http://localhost:8888/api/interpreter'.

Reach out to me on Instagram for a faster reply! "Buy Me A Coffee"

Machine Learning Applied to Stock Market Prediction: A Comparison between LSTM and ESN

5 minute read

Published:

I have been working with various neural networks for a while and always find recurrent neural network (RNN) very special. Whenever, there is a dependency with previous data points, then these networks shines out. Time-series problem especially fits here for e.g. predicting the weather pattern or stock market. In the past, I have done work to compare various RNNs for such tasks and my work concluded that ESN works quite well compared to simple deep neural network and some RNNs like LSTM or GRU (paper-1, paper-2 and paper-3) (of course this statement vary with the domain). Those works were done mostly with the scientific data for turbulent flow for thermal plumes, which has significant effects on weather (Wikipedia). Now, I was curious to see how these network can work for practical purpose.

Deep learning on HPC cluster with LSF queue

2 minute read

Published:

This is rather a short documentation to run the TensorFlow jobs on a high performance computing (HPC) cluster which is using LSF Load Sharing Facility. This can be extrapolated to other HPC systems with some tweaks. If you’re new to LSF and HPC jobs with DL then this post nicely summerizes the jargon.