Principal Data Engineer

7 days ago


Remote, India Mig Staffing Full time

Job Description :


We are seeking a dynamic and highly skilled Principal Data Engineer who has extensive experience building enterprise scale data platforms and lead these foundational efforts. This role demands someone who not only possesses a profound understanding of the data engineering landscape but is also at the forefront of their game. The ideal candidate will contribute significantly to platform development with diverse skillset while also being very hands-on coding and actively shaping the future of our data ecosystem.

Responsibilities :

- As a Architect/principal engineer, you will be responsible for ideation, architecture, design and development of new enterprise data platform. You will collaborate with other cloud and security architects to ensure seamless alignment within our overarching technology strategy.

- Architect and design core components with a microservices architecture, abstracting platform, and infrastructure intricacies.

- Create and maintain essential data platform SDKs and libraries, adhering to industry best practices.

- Design and develop connector frameworks and modern connectors to source data from disparate systems both on-prem and cloud.

- Design and optimize data storage, processing, and querying performance for large-scale datasets using industry best practices while keeping costs in check.

- Architect and design the best security patterns and practices

- Design and develop data quality frameworks and processes to ensure the accuracy and reliability of data.

- Collaborate with data scientists, analysts, and cross functional teams to design data models, database schemas and data storage solutions.

- Design and develop advanced analytics and machine learning capabilities on the data platform.

- Design and develop observability and data governance frameworks and practices.

- Stay up to date with the latest data engineering trends, technologies, and best practices.

- Drive the deployment and release cycles, ensuring a robust and scalable platform.

Requirements :

- 10+ of proven experience in modern cloud data engineering, broader data landscape experience and exposure and solid software engineering experience.

- Prior experience architecting and building successful enterprise scale data platforms in a green field environment is a must.

- Proficiency in building end to end data platforms and data services in GCP is a must.

- Proficiency in tools and technologies : BigQuery, Cloud Functions, Cloud Run, Dataform, Dataflow, Dataproc, SQL, Python, Airflow, PubSub.

- Experience with Microservices architectures - Kubernetes, Docker and Cloud Run

- Experience building Symantec layers.

- Proficiency in architecting and designing and development experience with batch and real time streaming infrastructure and workloads.

- Solid experience with architecting and implementing metadata management including data catalogues, data lineage, data quality and data observability for big data workflows.

- Hands-on experience with GCP ecosystem and data lakehouse architectures.

- Strong understanding of data modeling, data architecture, and data governance principles.

- Excellent experience with DataOps principles and test automation.

- Excellent experience with observability tooling : Grafana, Datadog.

Nice to have :

- Experience with Data Mesh architecture.

- Experience building Semantic layers for data platforms.

- Experience building scalable IoT architectures

(ref:hirist.tech)

  • Remote, India InOrg Global Full time

    **Job Description: Principal Data Scientist** **About InOrg Global**: InOrg Global helps organizations achieve global competitiveness through seamless integration of talent, technology, and strategic capabilities. As a Global Capability Center Partner to companies of all sizes, from startups to Fortune500s, InOrg Global augments clients capabilities to...


  • Remote/Bangalore, India The Principal TAG Full time

    About The Principal TAGWe are seeking an experienced Senior Data Quality Strategist to lead our COE establishment and centralize all testing artifacts using Jira TTM. The ideal candidate will have 15+ years of QA experience, proficiency in Tosca DI, test case management, and test automation strategies.Estimated Salary: $150,000 - $180,000 per yearJob...


  • Remote, India Merit Data and Technology Full time

    We are seeking a highly skilled and experienced Principal Machine Learning Developer to join our dynamic team. The ideal candidate will have a strong background in machine learning (ML), deep learning (DL), and large language models (LLMs). They should be proficient in Python and possess expertise in conventional ML algorithms, DL techniques,...


  • Remote, India The Principal TAG Full time

    About the Role: As a Senior Quality Assurance Automation Expert at The Principal TAG, you will have the opportunity to establish and lead our IT Testing Center of Excellence (COE). Responsibilities: - Establish a centralized testing framework using Jira TTM and Tosca DI. - Develop and enforce standardized guidelines for test case design, execution, and...

  • Principal Data Science

    3 months ago


    Remote, India ixceed solutions Full time

    **Role: Principal Data Science Engineer** **Exp: 10+ Years** **Location: Remote** **Notice Period: 30 Days Max** **Key Responsibilities**: - **Model Development and Implementation**: - Design, develop, and deploy advanced data science models and algorithms. - Ensure models are scalable, maintainable, and optimized for performance. - **Leadership and...


  • Remote, India The Principal TAG Full time

    Job Description:The Principal TAG is seeking a highly skilled Tricentis Tosca Automation Subject Matter Expert to join our team and contribute to the development and execution of robust test automation frameworks.About the Role:We are looking for an experienced professional with strong knowledge of functional test automation, Tricentis Tosca, PL/SQL, TOSCA...


  • Remote, India Premier Research Full time

    Premier Research is looking for a **Principal Biostatistician **to join our **Biostatistics t**eam! You will help biotech, MedTech, and specialty pharma companies transform life-changing ideas and breakthrough science into new medicines, devices, and diagnostics. What we do is profoundly connected to saving and improving lives, and we recognize our team...

  • Principal Architect

    2 months ago


    Remote, India NewPage Digital Healthcare Solutions Full time

    As a Principal Architect, you will own the integrity and strength of the solutions recommended, leveraging the transformative power of artificial intelligence to elevate our capabilities. You'll act as a trusted advisor to our team and clients, utili **Benefits**: Flexible and remote work Employee-centric culture Supportive peers Work-life balance Generous...

  • Principal Architect

    7 months ago


    Remote, India DeepHealth Full time

    DeepHealth is seeking a highly experienced and motivated Principal Architect for our Cloud PACS (Picture Archiving and Communication System) team. This role is pivotal in designing, developing, and implementing scalable, secure, and efficient cloud-based PACS solutions. The Principal Architect will work closely with cross-functional teams to ensure our...

  • Azure Data Architect

    7 months ago


    Remote, India Data PlatformExperts Full time

    Job Description: Azure Data Engineer (Architect Level) We are seeking an experienced and highly skilled Azure Data Engineer at the Architect level to lead the design and implementation of complex data solutions on the Microsoft Azure platform. As an Azure Data Engineer Architect, you will be responsible for providing technical expertise, guiding the...


  • Remote, India Caidya Full time

    **Job Title**:Principal** **Clinical SAS Programmer**Job Location**: India (Remote)**Job Overview**: The Principal Clinical SAS Programmer supports the Company’s data operations by providing programming services to Data Management. The Principal Clinical SAS Programmer provides technical and functional expertise in SAS, and where appropriate, mentors and...


  • Remote, India S&P Global Full time

    **Commodity Insights** **The **Role**:Geospatial Principal (SCAP Upstream 27) **The Location**:Hyderabad **Grade/Level **( _relevant for internal applicants only _): Geospatial Principal, IC7, 10A **The Team**:North American Land solutions in Upstream Data Operations & Applications from S&P Global Commodity Insights take a comprehensive look at Oil and...

  • Azure Data Architect

    7 months ago


    Remote, India Data PlatformExperts Full time

    **Job Description: Azure Data Architect** **Key Responsibilities**: - **Design and Implementation**: - Design and implement end-to-end data solutions (data models, pipelines, data lakes, data warehouses) in Azure. - Optimize and maintain existing Azure data structures and integration processes. - Ensure architectural solutions are scalable, maintainable,...


  • Remote, India Careator Technologies Pvt Ltd Full time

    Lead PL/SQL ExpertCareator Technologies Pvt Ltd seeks an accomplished Principal Database Developer and Architect to spearhead the design, development, and implementation of complex database solutions.About the Role:This is a highly specialized position that demands exceptional expertise in advanced PL/SQL concepts, data migration strategies, and Oracle...


  • Pune/Remote, India i2e Consulting Pvt Ltd Full time

    About the Role:">We are seeking a highly skilled Principal Data Science Architect to join our team at i2e Consulting Pvt Ltd. As an AI and ML Lead Specialist, you will play a crucial role in driving our data-driven initiatives and developing innovative solutions using machine learning algorithms.">Key Responsibilities:">">Leverage AI/ML techniques to...

  • Data Science

    7 months ago


    Remote, India Skillety Technologies Full time

    **Responsibilities**: - Drive development, scaling and execution of ML/AI models for Dell internal business function - Design, develop and program methods, processes and systems to consolidate and analyze structured and unstructured data from multiple sources to generate actionable insights and solutions. - Produce actionable and compelling recommendations...


  • Remote, India Mig Staffing Full time

    Mig Staffing is seeking a skilled Principal Data Engineer to lead the development of our enterprise-scale data platforms.Job Description:We are looking for a highly experienced professional with extensive knowledge of modern cloud data engineering and software engineering. The ideal candidate will possess a profound understanding of the data engineering...

  • Principal Architect

    7 months ago


    Remote, India Game Changer IT Services Full time

    **Principal Architect - 15 + years - Haryana, Gurgaon Location** **Skills : Java, Node.js, PHP, ApacheKafka, AWS** **Role & Responsibilities** - Design and build highly scalable, reliable and available software products. - Your ability to partner with business, product teams will set you apart. - Work closely with product and business teams towards...


  • Remote, India BlueByte Technologies Full time

    Location : Remote PositionDuration : 12 Months with possible extensionEmployment Type : C2CClient : Zendesk (They will be working in USA timings EST zone till 2:00 PM, as per India timings working hours will be 2:30 PM to 11:30 PM IST.)Location preference : PAN India Skills Needed :- Data transformation (DBT)- Data curation Job Description : A data curation...


  • Remote, India The Principal TAG Full time

    Role : IT Testing CoE/Practice Lead. Location : Remote Work. Duration : Long Term.Work Schedule : Flexible. Rates/Salary : Negotiable, DOE. Responsibilities :- COE Establishment: Build the COE from scratch, developing templates, testing standards, and methodologies for efficient and accurate data testing.- Centralization: Lead efforts to centralize all...