Big Data Infrastructure Lead

2 weeks ago

Meerut, Uttar Pradesh, India OSD Data Services Full time

Role Description

">

As a Data Platform Engineer at OSD Data Services, you'll be responsible for designing, building, and optimizing high-performance data platforms that enable businesses to extract maximum value from their data. With a strong focus on innovation and collaboration, our team thrives on solving complex data challenges.

You'll leverage your expertise in open-source technologies to architect and implement scalable data lakehouse platforms, develop and optimize data pipelines, and manage deployment and orchestration using Kubernetes and Docker.

">

Key Responsibilities

">
  • Data Engineering Expertise: Leverage your expertise in open-source technologies to architect and implement scalable data lakehouse platforms.
  • Pipeline Optimization: Develop and optimize data pipelines for real-time and batch processing, utilizing tools like Spark and Airflow.
  • Storage and Format Management: Implement modern storage formats such as Apache Iceberg, Hudi, or Delta Lake, ensuring efficient data storage and retrieval.
  • Deployment and Orchestration: Manage deployment and orchestration using Kubernetes and Docker across cloud or on-prem environments.
  • Data Governance: Establish and maintain robust governance practices, including database CDC replication and lineage tracking.
  • Analytics Integration: Support interactive dashboards and analytics tools, such as Superset or similar, enabling data-driven decisions.
  • Performance Monitoring: Monitor system performance and implement optimizations to ensure scalability and cost-efficiency.
">

Requirements

">
  • 3+ years of experience in data platform engineering, with expertise in open-source tools and cloud technologies.
  • Strong understanding of distributed computing frameworks like Spark and query engines like Trino or Presto.
  • Proficiency in containerization and orchestration technologies (Docker, Kubernetes).
  • Hands-on experience with cloud platforms (AWS, GCP, Azure) and hybrid/on-prem infrastructure.
  • Familiarity with ETL/ELT workflows, CDC replication tools, and modern data storage solutions.
  • Proficiency in programming languages like Python, Java, or Scala.
  • Experience managing large-scale data systems (terabyte/petabyte-scale) and optimizing for performance.
  • Excellent problem-solving skills and a passion for leveraging open-source technologies to build transformative solutions.
">

Preferred Qualifications

">
  • Familiarity with AI/ML frameworks and integration.
  • Experience with CI/CD workflows and infrastructure as code (e.g., Terraform, Ansible).
  • Knowledge of data governance frameworks and compliance standards.
">

What We Offer

">
  • Competitive Compensation: $140,000 - $220,000 per year.
  • Flexible Work Options: Remote work flexibility and a healthy work-life balance.
  • Ongoing Learning Opportunities: We invest in your professional development, offering learning opportunities and career progression pathways.
  • Collaborative Environment: Join a supportive team of experts who thrive on solving complex challenges together.
">

$160,000 - $240,000 per year

  • Big Data Engineer Specialist

    2 weeks ago

    Meerut, Uttar Pradesh, India OSD Data Services Full time

    We are seeking an experienced Big Data Engineer Specialist to join our team at OSD Data Services. As a specialist, you will be responsible for designing, building, and optimizing scalable, high-performance data platforms using cutting-edge technologies like Apache Spark, Kafka, and Kubernetes. This role requires strong technical expertise, excellent...

  • Big Data Engineer

    2 weeks ago

    Meerut, Uttar Pradesh, India OSD Data Services Full time

    About UsAt OSD Data Services, we're revolutionizing the way businesses leverage their data. Our mission is to empower organizations with seamless, unified, and scalable data infrastructure that enables smarter decision-making and faster innovation.We pride ourselves on being at the forefront of data engineering, combining a passion for innovation with a...

  • Senior Data Infrastructure Architect

    2 weeks ago

    Meerut, Uttar Pradesh, India OSD Data Services Full time

    About OSD Data ServicesWe're a cutting-edge data engineering company redefining how businesses leverage their data. Our mission is to empower organizations with seamless, unified, and scalable data infrastructure for smarter decision-making and faster innovation.We combine passion for innovation with a commitment to solving real-world data challenges. By...

  • Data Infrastructure Architect

    2 weeks ago

    Meerut, Uttar Pradesh, India OSD Data Services Full time

    Innovative WorkAs a Data Platform Engineer at OSD Data Services, you'll be part of a team that's shaping the future of data infrastructure. You'll work with open-source technologies at scale, tackling complex challenges in data engineering and analytics.You'll collaborate with talented teams to build data platforms that deliver real-time insights, support...

  • Cloud Infrastructure Architect

    2 weeks ago

    Meerut, Uttar Pradesh, India OSD Data Services Full time

    About OSD Data ServicesAt OSD Data Services, we're redefining how businesses leverage their data. Our mission is to empower organizations with a seamless, unified, and scalable data infrastructure that enables smarter decision-making and faster innovation. With a focus on cloud-based technologies, we've built a comprehensive Data Lakehouse Platform that...

  • Senior Big Data Engineer

    2 weeks ago

    Meerut, Uttar Pradesh, India OSD Data Services Full time

    About Our Tech StackYou'll work with a comprehensive suite of tools and frameworks, including:Distributed computing: Apache Spark, HiveStreaming and messaging: KafkaQuery engines: Trino, PrestoOrchestration: AirflowStorage formats: Apache Hudi, Iceberg, Delta LakeVisualization: SupersetContainerization: Docker, KubernetesCloud platforms: AWS, GCP, AzureAbout...

  • Data Infrastructure Optimization Expert

    2 weeks ago

    Meerut, Uttar Pradesh, India OSD Data Services Full time

    Key ResponsibilitiesDesign and Development: Architect and implement scalable Data Lakehouse Platforms using open-source technologies.Pipeline Optimization: Develop and optimize data pipelines for both real-time and batch processing with tools like Spark and Airflow.Storage and Format Management: Implement modern storage formats such as Apache Iceberg, Hudi,...

  • Big Data Engineer and Innovator

    2 weeks ago

    Meerut, Uttar Pradesh, India OSD Data Services Full time

    What We OfferAs a Data Platform Engineer at OSD Data Services, you can expect:A Competitive Salary: $140,000 - $180,000 per year, depending on experience.Opportunities for Growth: Our team is passionate about innovation and continuous learning, providing opportunities for professional growth and development.A Collaborative Environment: Join a dynamic and...

  • Data Infrastructure Developer

    2 weeks ago

    Meerut, Uttar Pradesh, India OSD Data Services Full time

    About Our TeamWe are a team of experts in data engineering, committed to delivering innovative data solutions that drive real-world results for businesses across industries. Our team members are passionate about staying up-to-date with the latest technologies and best practices, ensuring our clients receive the best possible outcomes.We're looking for a...

  • Senior Data Architect

    2 weeks ago

    Meerut, Uttar Pradesh, India OSD Data Services Full time

    About OSD Data ServicesOSD Data Services is a trailblazer in the data engineering landscape, pushing the boundaries of what's possible with data. Our vision is to empower organizations with unparalleled data insights, driving informed decision-making and business growth. With a focus on innovative data lakehouse solutions, we've developed a robust platform...

  • Cloud Data Architect

    2 weeks ago

    Meerut, Uttar Pradesh, India OSD Data Services Full time

    About UsAt OSD Data Services, we're revolutionizing data engineering by empowering organizations with unified, scalable, and seamless data infrastructure.We pride ourselves on being at the forefront of cutting-edge technologies, combining a passion for innovation with a commitment to solving real-world data challenges. As a Data Platform Engineer, you'll...

  • Enterprise Data Architect

    2 weeks ago

    Meerut, Uttar Pradesh, India OSD Data Services Full time

    Unlock the Full Potential of Your Business with OSD Data ServicesAt OSD Data Services, we're dedicated to empowering organizations with a seamless, unified, and scalable data infrastructure that enables smarter decision-making and faster innovation.We're seeking an experienced Data Platform Engineer with a passion for open-source technologies to join our...

  • Data Engineering Specialist

    2 weeks ago

    Meerut, Uttar Pradesh, India OSD Data Services Full time

    About OSD Data ServicesAt OSD Data Services, we're redefining the way businesses harness their data potential. Our mission is to empower organizations with a seamless, unified, and scalable data infrastructure that fosters smarter decision-making and accelerated innovation. With a focus on cutting-edge data lakehouse technologies, we've developed a...

  • Data Engineering Expert

    2 weeks ago

    Meerut, Uttar Pradesh, India OSD Data Services Full time

    About OSD Data ServicesWe are a cutting-edge data engineering company that empowers businesses to unlock the full potential of their data. Our mission is to deliver seamless, unified, and scalable data infrastructure that drives smarter decision-making and faster innovation.Role OverviewWe are seeking a highly skilled Data Platform Engineer to design, build,...

  • Senior Data Solutions Architect

    2 weeks ago

    Meerut, Uttar Pradesh, India OSD Data Services Full time

    About OSD Data ServicesWe are a cutting-edge data engineering company, redefining how businesses leverage their data. Our mission is to empower organizations with seamless, unified, and scalable data infrastructure that enables smarter decision-making and faster innovation.

  • Data Engineer Specialist

    2 weeks ago

    Meerut, Uttar Pradesh, India OSD Data Services Full time

    Company Overview">At OSD Data Services, we're revolutionizing the way businesses harness their data. Our mission is to empower organizations with a seamless, unified, and scalable data infrastructure that enables informed decision-making and accelerated innovation.We pride ourselves on being at the forefront of data engineering, combining a passion for...

  • Data Engineer with Passion

    2 weeks ago

    Meerut, Uttar Pradesh, India OSD Data Services Full time

    Are you ready to take your data engineering career to the next level? We're looking for a talented Data Platform Engineer to join our team at OSD Data Services.About the RoleThis is an exciting opportunity to work on complex, petabyte-scale projects and shape the future of data infrastructure for businesses across industries.You'll design, build, and...

  • Cloud Data Architecture Expert

    2 weeks ago

    Meerut, Uttar Pradesh, India OSD Data Services Full time

    Overview">OSD Data Services is at the forefront of data engineering, pushing the boundaries of what's possible with data. As a Data Platform Engineer, you'll be instrumental in designing, building, and optimizing high-performance data platforms that transform businesses across industries.With a strong focus on innovation and collaboration, our team thrives...

  • Cloud Data Architect

    2 weeks ago

    Meerut, Uttar Pradesh, India OSD Data Services Full time

    A New Era in Data EngineeringOSD Data Services is revolutionizing the way businesses approach data engineering. We're looking for a talented Data Platform Engineer to join our team and contribute to the development of our comprehensive Data Lakehouse Platform.Job DescriptionThis role involves designing, building, and optimizing high-performance data...

  • Cloud-First Data Platform Leader

    2 weeks ago

    Meerut, Uttar Pradesh, India OSD Data Services Full time

    Your OpportunityThis role offers a unique chance to work on complex, petabyte-scale projects shaping the future of data infrastructure. You'll collaborate with talented teams to build data platforms delivering real-time insights, supporting advanced analytics, and integrating seamlessly with AI technologies.We invest in your professional development,...