The Senior Data Engineering Consultant has the following responsibilities and duties:
- Create and maintain optimal data architecture pipeline.
- Perform root cause analysis on internal and external data, processes and share insights with different stakeholders using reporting and visualizations tools.
- Design infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources into Snowflake and other platforms.
- Understand existing implementation and should enhance it as needed.
- Work with other data engineers, data ingestion specialists, and experts across cross-functional teams.
- Work on large datasets to meet functional and business requirements.
The Data Engineer Consultant should have the following skills, education and experience:
- Excellent SQL and Python skills
- Experience with Spark programming language(s)
- Experience with Docker/containerization
- Expertise in designing efficient Data Models
- Experience with ETL/ELT and familiarity of Data Warehouse concepts
- Experience with Snowflake Data Warehouse
- Strong knowledge across Microsoft Azure Tools.
- Data pipeline and workflow management tools: Databricks (Spark + Python), ADF, Dataflow
- Knowledge of API-based integration and security (Restful and SOAP)
- Experience with Data Visualization tools such as Power BI etc.
- Ability to learn fast and translate data into actionable results
- Excellent written and communication skills with client stakeholders
- Experience with JSON Programming
- Experience developing and supporting scalable data pipelines
- Solid Linux programming skills
- Cosmos DB, Teradata, SQL Server (all nice to have)
- Stream-processing systems: Streaming-Analytics, IoT Hub, Event Hub, Kusto Queries (all nice to have)
- Bachelors in computer science, engineering, information technology or other related disciplines.
- Masters or PhD in computer science, engineering, information technology or other related disciplines.
- An experienced professional will have at least 6+ years of professional experience in Data Engineering role with a Masters Degree or 8+ years of experience in a Data Engineering related role with a Bachelors degree.
- Citizenship: Not Provided
- Incentives: Not Provided
- Education: Not Provided
- Travel: Not Provided
- Telework: Not Provided