Data hub graphql-server
Graphql server for the data-hub application.
Dataware house & Github CMS integration
This project has a dataware house module thats responsible for getting data from the datawarehouse and merge it with refrence data from a github repo acting as a CMS. The github repo (cms) contains refrence files such as colors, country names etc.
This data is served over a graphql API.
npm run build - build the library files (Required for start:watch) npm start - Start the server npm run dev - Start the server in watchmode (Useful for development) npm test - run tests once npm run deploy - will deploy app to now.sh make sure you have the DB access credentials in your system env file eg .bash_profile
How to run it:
npm install -g yarn # if you dont have yarn installedyarn # install dependencies## development modenpm run dev## run in productionnpm start
- When you install a new dependency add its types from definetly typed typescript incase it doesnt come with any
yarn add --dev @types/ramda
- Whenever you change the .gql files run
npm run gqlToTsso that resolvers and type definitions are generated for use in graphql
On data caching
Every new instance of this node app starts with a fresh LRUcache which will be populated by items that get requested through the lifecycle of the application.
The app also persists queries on each data requests and does pre-caching of these queries on app boot.
During development; inorder to keep having fresh data, every data request is added to a queue as it gets served from the cache. This queue is ran in the background and updates after an hour of the request with new data if any.
.cache file contains all the queries occuring in the life time of the app. Its in git history so that we can precache those queries on app boot. (TODO: Make this run in another process)
We are currently test deploying with now and they freeze inactive normal deployments. In order to create a deployment that's always active consider scaling your deploy
now scale my-deployment-ntahoeato.now.sh 1
This repo is published to npm so that we can reuse its types
- remove console.info & console.error and replace with function that logs to an info & error file
- fix testing on travis
- Make uid immutable for testing purposes
- pull in di_id as id in sql queries
- Perf: breakdown tabs data fetching function into individual functions