Title:Sr Data Engineer
Duration:Right to Hire
Location:Eden Prairie, MN
IDEALBACKGROUND: We a looking for somebody who isvery comfortable working withdata with background of automating dataingestion, acquisition andtransformation and making data available in privateor public cloud. We don’trequire any industry specific background like healthcarethe project work ispretty much about infrastructure automaton and not domainspecific.
– Experiencebuildingreal time and batch data ingestion pipelines and ETL jobs using any ofthefollowing languages like Python, Java, Scala and tools like Logstash,ApacheBeam, Kafka, Kafka Streams, KSQL, Apache Airflow
– Goodknowledgeof ELK stack Elasticsearch, Logstash, Kibana
– Experiencewithcontainers and container orchestration platforms like Docker, Openshift,Kubernetes
– Buildscalable,fault-tolerant batch and real-time data ingestion pipelines, datatransformationand data mining jobs
– Createandmaintain data-driven APIs to support a wide range of data integrations anddataservices
– Workwithvarious data stores and databases including all aspects configurationandadministration required to maintain it
– Recommendandimplement best practices for data management and governance
– WorkwithData Scientist on various data acquisition, data preparation, Clientmodelmanagement and other Client engineering tasks
– Helpsettechnical direction and provide guidance to more junior data engineers.
– Ownorassist with incident and problem management
– Collaboratewiththe rest of the agile team at daily stand ups and other agile ceremonies
Whatskills/attributes are a must have?
? 5 years of experience with software development using with demonstratedprogression and positive career growth
? 2 years of experience with professional data engineering, building and using datainfrastructure, APIs, and integrations
? 2 years of experience with Docker, Kubernetes, or Openshift
? 2years of experience developing data pipelines, ETL jobs using languageslikePython OR Java OR Scala
? 1year of experience working knowledge of ELK stack and data streaming usingKafka
? Hassoundknowledge of data structures, schemas and algorithms
Whatskills/attributes are nice to have?
? ExperiencewithNeo4j or other graph databases
? Experiencewithdata streaming and data mining
? Experiencewithrelational and NoSQL databases
? Aptitudeforlearning and be a great team player – strong track record
? ExperiencewithAI, Machine Learning, NLP and/or predictive modeling