Comment on page
Data Handler
Data Handler is an open-sourced and extensible NodeJS server application that facilitates ETL processes for the data needs of SpatialWork. The source code for a template Data Handler is provided to all users to allow customization for any data requirements. You can find the template repository on Github. It allow you to fetch and save data to a MongoDB database and serve it via API.
- Node.js
- MongoDB Database
- 1.Fork this repository to your own GitHub account.
- 2.Clone the forked repository to your local machine.git clone https://github.com/<YOUR_USERNAME>/mysql-fetch-sample.git
- 3.Install the required nodejs dependencies.npm install
- 1.Define your job name, e.g. my_new_job
- 2.Create a folder in
jobs/
and create a newmy_new_job.js
file inside.- Define your job logic. See any of the job samples for reference.
- 3.Add a new job definition in
jobs/jobs.js
, in thejobs
array:{name: path.join('my_folder', 'my_new_job'),timeout: 0,interval: '1h',} - 4.Create the same folder in
model/
and create a newmodel-my_new_job.js
file inside.- Define your MongoDB collection schema. See any of the model samples for reference.
- 5.Add a new model definition in
models/models.js
. If your collection stores data in rows, add it torowDataModels
. If your collection stores data in snapshots, add it tosnapshotDataModels
.my_new_job: require('./my_folder/my_new_job'),
For any issues or support requests, please create an issue on the repository.
Last modified 4mo ago