Docker

Coming soon … -- -- -- DESCRIBE THAT WE HAD TO COME UP WITH SOME WAY TO DO -- WRITE A LITTLE BIT ABOUT BATCH PROCESSING VS STREAM PROCESSING -- -- -- -- DESCRIPTION OF LUIGI VS AIRFLOW VS ... -- DIRECTED ACYCLIC GRAPHS -- CONTAINER ORCHESTRATION (ECS, FARGATE, SOME WORDS ON KUBERNETES AND WHY WE DONT USE IT) -- EC2 INSTANCE CONFIGURATION (C4 INSTANCES) -- THE COSTS PER SESSION -- SCALING APPROACH, SPOT INSTANCES -- WE IMPLEMENTED PROTOTYPE VERSIONS OF OUR ALGORITHMS IN C++ AND -- WE TRANSLATED A KALMAN SMOOTHER FROM PYTHON TO C++ ARMADILLO -- WE TRANSLATED OUR IMU PREPROCESSOR USING CYTHON -- -- -- -- VERSION CONTROL WORKFLOW (ONEFLOW) -- VERSIONING APPROACH -- SCREENSHOT OF JUPYTER NOTEBOOK EVALUATION -- -- DESCRIBE THE FACTOR GRAPH ALGORITHM AND PREPROCESSING -- DESCRIBE OFFLINE VS ONLINE ESTIMATION -- ADD FACTOR GRAPH SKETCH -- SIMPLE IMU KITTI EXAMPLE FACTOR GRAPH -- -- WE TEAMED UP WITH SENSOR FUSION EXPERTS FROM KNOWTION -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --

The TRACKTICS Pipeline is a framework that allows to orchestrate parallel DAG (Direct Acyclic Graphs) batch processing jobs. It is based on Docker for artifact deployment, ECS (Amazon EC2 Container Service) for container orchestration and resource discovery, Luigi (by Spotify) for workflow and dependency management, SQS (Amazon Simple Queuing System) as a job queue, a custom service for auto scaling, S3 and Postgres (Amazon RDS) to store results and Kinesis for logging.