Note: The job is a remote job and is open to candidates in USA. Sapphire Software Solutions Inc is recruiting a Senior Azure Data Engineer for an unnamed client. The role supports GenAI solutions for enterprise products by building and maintaining large-scale analytics and data pipelines, including ingestion, streaming, batch processing, ETL, and SQL performance tuning.

Responsibilities

• Responsible for the building, deployment, and maintenance of mission-critical analytics solutions that process terabytes of data quickly at big-data scales
• Contributes design, code, configurations, manage data ingestion, real-time streaming, batch processing, ETL across multiple data storages
• Responsible for performance tuning of complicated SQL queries and Data flows

Skills

• At least 9+ years of experience required
• The candidate must have excellent communication skills
• Experience coding in SQL/Python(\•), with solid CS fundamentals including data structure and algorithm design
• Hands-on implementation experience working with a combination of the following technologies: Hadoop, Map Reduce, Kafka, Hive, Spark(\•), SQL and NoSQL data warehouses (\•)
• Experience in Azure cloud data platform(\•)
• Must have: understanding of LLM Pipelines, including data preprocessing for GenAI Models
• Experience coding in SQL/Python(\•), with solid CS fundamentals including data structure and algorithm design
• Hands-on implementation experience working with a combination of the following technologies: Hadoop, Map Reduce, Kafka, Hive, Spark(\•), SQL and NoSQL data warehouses (\•)
• Experience in Azure cloud data platform(\•)
• Experience working with vector databases (Milvus, Postgres, etc.) (\•)
• Knowledge of embedding models and retrieval-augmented generation (RAG) architectures
• Understanding of LLM pipelines, including data preprocessing for GenAI models (\•)
• Experience deploying data pipelines for AI/ML workloads(\•), ensuring scalability and efficiency
• Familiarity with model monitoring(\•), feature stores (Feast, Vertex AI Feature Store), and data versioning(\•)
• Experience with CI/CD for ML pipelines(\•) (Kubeflow, MLflow, Airflow, SageMaker Pipelines)
• Understanding of real-time streaming for ML model inference (Kafka, Spark Streaming) (\•)
• Knowledge of Data Warehousing, design, implementation and optimization
• Knowledge of Data Quality testing, automation and results visualization
• Knowledge of BI reports and dashboards design and implementation (PowerBI)(\•)
• Experience with supporting data scientists and complex statistical use cases

Benefits

• 100% remote, with the candidate required to sit in the EST/CST time zone
• Opportunity to build a GenAI solution embedded within the platform and work on the latest technology for an industry tax solution leader

Company Overview

Company H1B Sponsorship