Note: The job is a remote job and is open to candidates in USA. The Transformation Group is seeking a Senior Data Engineer to build a new AWS data platform for a client and lead the migration of existing workloads from Azure to AWS. The role will develop a medallion-layer lakehouse, analytics and reporting models, and data foundations for AI solutions.

Responsibilities

• Build the AWS data platform: S3 lakehouse on open table formats (Iceberg/Delta) organized in medallion (bronze/silver/gold) layers, with a Redshift, Snowflake, or Databricks serving layer
• Migrate Azure workloads (Data Factory, Synapse, Azure SQL, ADLS) to AWS: assess, re-platform pipelines and models, validate data parity, and support cutover
• Build batch and event-driven pipelines (Glue, Step Functions, Lambda, Kinesis) from Salesforce and other enterprise sources
• Develop gold-layer star-schema models that support Tableau and self-service analytics
• Build the data foundation for AI: vector pipelines, RAG, and agentic/LLM solutions using Amazon Bedrock, SageMaker, and OpenSearch
• Deliver infrastructure as code and CI/CD (Terraform/CDK, Git) with governance, security, and data-quality controls
• Tune and cost-optimize workloads; document architecture and migration mappings; report progress to stakeholders

Skills

• Excellent communication skills and ability to communicate with our clients in the US
• 7+ years of experience with data engineering
• 5+ years running production data platforms on AWS; with hands-on experience on AWS data stack: S3, Glue, Redshift, Athena, Lambda, Step Functions, Lake Formation, IAM
• 5+ years of Snowflake and/or Databricks experience building medallion-layer lakehouses
• Hands-on Snowflake and/or Databricks development (Spark/SQL transformations, warehouse and lakehouse models)
• Hands-on experience with delivering AI/ML or GenAI solutions to production (e.g., RAG app, AI agent, ML pipeline)
• Experience with Azure data services (Data Factory, Synapse, Azure SQL, ADLS)
• Data modeling: conceptual, logical, and physical design; dimensional (star/snowflake schema) modeling with conformed dimensions and slowly changing dimensions; normalized and Data Vault patterns; defining grain, keys, and semantic-layer models for BI
• Modern techniques: medallion architecture, ELT, change data capture, incremental loads, dbt, data-quality testing
• Cloud architecture: secure, resilient, cost-efficient design (VPC, identity, encryption, multi-account)
• Applied AI/GenAI: Bedrock or SageMaker, LLM integration, RAG, vector databases
• Strong Python (PySpark) and advanced SQL; Parquet, Iceberg, and Delta formats
• Ability to handle sensitive data and communicate technical concepts to executives and non-technical staff
• Bachelor's in Computer Science or related field, or equivalent
• Strong experience with cloud migration; Azure to AWS preferred
• Salesforce integration and Tableau is a plus
• Agent frameworks/MCP a plus
• AWS certification preferred

Company Overview

Company H1B Sponsorship