Listing Description

Job Description

The Senior Data Engineering Consultant has the following responsibilities and duties: 

• Create and maintain optimal data architecture pipeline.


• Perform root cause analysis on internal and external data, processes and share insights with different stakeholders using reporting and visualizations tools.


• Design infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources into Snowflake and other platforms.


• Understand existing implementation and should enhance it as needed.


• Work with other data engineers, data ingestion specialists, and experts across cross-functional teams.


• Work on large datasets to meet functional and business requirements.

Requirements

The Data Engineer Consultant should have the following skills, education and experience: 

Skills  

      Required:

• Excellent SQL and Python skills


• Experience with Spark programming language(s)


• Experience with Docker/containerization


• Expertise in designing efficient Data Models


• Experience with ETL/ELT and familiarity of Data Warehouse concepts


• Experience with Snowflake Data Warehouse


• Strong knowledge across Microsoft Azure Tools.


• Data pipeline and workflow management tools: Databricks (Spark + Python), ADF, Dataflow


• Knowledge of API-based integration and security (Restful and SOAP)


• Experience with Data Visualization tools such as Power BI etc.


• Ability to learn fast and translate data into actionable results


• Excellent written and communication skills with client stakeholders

      Preferred:

• Experience with JSON Programming


• Experience developing and supporting scalable data pipelines


• Solid Linux programming skills


• Cosmos DB, Teradata, SQL Server (all nice to have)


• Stream-processing systems: Streaming-Analytics, IoT Hub, Event Hub, Kusto Queries (all nice to have)

Education

      Required:

• Bachelors in computer science, engineering, information technology or other related disciplines.

      Preferred:

• Masters or PhD in computer science, engineering, information technology or other related disciplines. 

Experience

• An experienced professional will have at least 6+ years of professional experience in Data Engineering role with a Masters Degree or 8+ years of experience in a Data Engineering related role with a Bachelors degree.

#LI-REMOTE