Comment on page

Data Handler


Data Handler is an open-sourced and extensible NodeJS server application that facilitates ETL processes for the data needs of SpatialWork. The source code for a template Data Handler is provided to all users to allow customization for any data requirements. You can find the template repository on Github. It allow you to fetch and save data to a MongoDB database and serve it via API.


  • Node.js
  • MongoDB Database


  1. 1.
    Fork this repository to your own GitHub account.
  2. 2.
    Clone the forked repository to your local machine.
    git clone<YOUR_USERNAME>/mysql-fetch-sample.git
  3. 3.
    Install the required nodejs dependencies.
    npm install


Add a new job

  1. 1.
    Define your job name, e.g. my_new_job
  2. 2.
    Create a folder in jobs/ and create a new my_new_job.js file inside.
    • Define your job logic. See any of the job samples for reference.
  3. 3.
    Add a new job definition in jobs/jobs.js, in the jobs array:
    name: path.join('my_folder', 'my_new_job'),
    timeout: 0,
    interval: '1h',
  4. 4.
    Create the same folder in model/ and create a new model-my_new_job.js file inside.
    • Define your MongoDB collection schema. See any of the model samples for reference.
  5. 5.
    Add a new model definition in models/models.js. If your collection stores data in rows, add it to rowDataModels. If your collection stores data in snapshots, add it to snapshotDataModels.
    my_new_job: require('./my_folder/my_new_job'),

Additional Resources


For any issues or support requests, please create an issue on the repository.