Title: Data Engineer with API, ETL, Kafka
Location: Eden Prairie, MN, USA
Length: 5 Months
Restriction: W2 or C2C
***Onsite-No Remote*** webcam interview;
- Experience building real time and batch data ingestion pipelines and ETL jobs using any of the following languages like Python, Java, Scala and tools like Log stash, Apache Beam, Kafka, Kafka Streams, KSQL, Apache Airflow
- Good knowledge of ELK stack Elastic search, Log stash, Kibana
- Experience with containers and container orchestration platforms like Docker, Open shift, Kubernetes
- Build scalable, fault-tolerant batch and real-time data ingestion pipelines, data transformation and data mining jobs
- Create and maintain data-driven APIs to support a wide range of data integrations and data services
- Work with various data stores and databases including all aspects configuration and administration required to maintain it
- Recommend and implement best practices for data management and governance
- Work with Data Scientist on various data acquisition, data preparation, ML model management and other ML engineering tasks
- Help set technical direction and provide guidance to more junior data engineers.
- Own or assist with incident and problem management
- Collaborate with the rest of the agile team at daily stand ups and other agile ceremonies
- 5+ years of experience with software development using with demonstrated progression and positive career growth
- 2+ years of experience with professional data engineering, building and using data infrastructure, APIs, and integrations
- 2+ years of experience with Docker, Kubernetes, or Open shift
- 2+ years of experience developing data pipelines, ETL jobs using languages like Python OR Java OR Scala
- 1+ year of experience working knowledge of ELK stack and data streaming using Kafka Has sound knowledge of data structures, schemas and algorithms.
Nice to have
- Experience with Neo4j or other graph databases Experience with data streaming and data mining Experience with relational and NoSQL databases Aptitude for learning and be a great team player – strong track record
- Experience with AI, Machine Learning, NLP and/or predictive modeling.