Note: The job is a remote job and is open to candidates in USA. Trility Consulting is seeking a Lead Data Consultant to modernize a client’s Azure Synapse platform and migrate prioritized workloads to Microsoft Fabric. The role involves refining the Fabric architecture, building reusable ingestion frameworks and pipelines, validating migrations, and establishing CI/CD workflows while collaborating hands-on with client engineers.

Responsibilities

• Evaluate and refine the existing Microsoft Fabric architecture, including metadata structures, table-level event triggering, and reusable ingestion patterns
• Design and implement a metadata-driven ingestion framework that allows new tables and sources to be onboarded through configuration rather than custom development
• Migrate prioritized data sources and workloads from Azure Synapse into Microsoft Fabric
• Develop and orchestrate Fabric data pipelines and Spark notebooks using PySpark and Spark SQL
• Analyze existing Azure Synapse pipelines, including complex and nested workflows, to understand dependencies and translate existing processing logic into Fabric solutions
• Build data validation and reconciliation processes to verify migration accuracy, including duplicate detection, key alignment, and record-level comparisons between Synapse and Fabric
• Analyze source-specific loading requirements and recommend appropriate full-load, incremental, or change data capture (CDC) patterns
• Implement Git-based CI/CD and automated environment promotion for Fabric artifacts, including pipelines and notebooks, using GitHub Actions, Azure DevOps, or similar tooling
• Leverage Claude Code to define implementation specifications, accelerate development, and review AI-generated code for quality, correctness, and maintainability
• Evaluate data dictionary, metadata, and lineage requirements, including potential catalog solutions such as Microsoft Purview
• Collaborate with client engineers through paired development, code reviews, and knowledge-sharing sessions to establish sustainable engineering standards
• Apply data engineering best practices related to performance, testing, monitoring, documentation, security, and operational support

Skills

• Candidates must respect and value people, recognize that over-communication is barely good enough, thrive on solving complex problems, have a passion for building teams, and know that delivering what a client actually values is more important than their own predispositions
• They understand value proposition, love delivering value, and take pride in learning the expansive and ever-changing business of clients
• Candidates are self-motivated and relentlessly working to become more today than they were yesterday
• Experience using Claude Code as part of an AI-assisted development workflow is essential to this engagement
• 5+ years of experience working with Azure Data Services, including Microsoft Fabric, Azure Data Factory, Databricks, or Synapse
• Hands-on Microsoft Fabric experience in production environments, including lakehouses, pipelines, notebooks, medallion architecture, workspaces, and capacity management
• Strong experience building and maintaining production ETL/ELT pipelines and reusable data ingestion frameworks
• Demonstrated experience designing and implementing a metadata-driven ingestion framework, including configuration-based onboarding and extensible processing patterns
• Experience developing Spark notebooks and data transformation workflows using PySpark and Spark SQL
• Strong SQL skills for data transformation, validation, and performance tuning, including the ability to read and understand existing T-SQL logic
• Experience working with Azure Synapse, including analyzing existing pipelines and understanding complex data processing dependencies
• Experience implementing medallion architecture patterns (Bronze, Silver, Gold)
• Experience designing and implementing Git-based CI/CD workflows for data platform artifacts using GitHub Actions, Azure DevOps, or similar tooling
• Hands-on experience using Claude Code for AI-assisted software development, including directing code generation, working across multiple files, and critically reviewing generated code
• Experience validating data migrations and ensuring data quality, accuracy, and consistency across source and target systems
• Experience integrating data from REST APIs, relational databases, and structured or semi-structured data sources
• Familiarity with data governance, lineage, metadata management, and data catalog concepts
• Ability to collaborate effectively with technical teams, challenge existing approaches constructively, and participate in client-facing technical discussions
• Strong written and verbal communication skills
• Must be authorized to work in the United States without sponsorship now or in the future
• Experience migrating data platforms or workloads from Azure Synapse to Microsoft Fabric
• Experience working with Oracle and DB2 data sources
• Experience implementing incremental ingestion, change data capture (CDC), and source-specific loading strategies
• Experience developing data reconciliation and validation frameworks for large-scale migrations
• Experience evaluating Microsoft Purview or other data catalog and lineage solutions
• Experience working within consulting or professional services environments

Benefits

• Work remotely anywhere in the United States with flexible work hours
• Paid time-off, company holidays, and floating holidays
• Comprehensive benefits with contributions toward premiums
• 401(k) plan
• Sponsored and supported learning opportunities

Company Overview