Data Engineer

1 week ago

Delhi, India Test Triangle Full time
Job DescriptionData Engineer

As a Principal Data Engineer, your responsibilities will include:

Design and build data pipelines to process terabytes of data

Orchestrate in Airflow the data tasks to run on Kubernetes/Hadoop for the ingestion, processing and cleaning of data.

Create Docker images for various applications and deploy them on Kubernetes

Design and build best in class processes to clean and standardize data.

Troubleshoot production issues in our Elastic Environment

Tuning and optimizing data processes

·

Advancing the team’s DataOps

culture (CI/CD, Orchestration, Testing, Monitoring) and building out standard development patterns

Drive innovation

by testing new technology and approaches to continually advance the capability of the data engineering function.

Drive efficiencies

in current engineering processes via standardization and migration of existing on-premise processes to the cloud

Ensuring Data Quality –

building best in class data quality monitoring that ensure that all data products exceed customer expectations.

Required Qualifications:

Computer Science bachelor’s degree or similar.

Good understanding of Data Modelling techniques i.e. DataVault, Kimble Star

Excellent understanding of Column-Store RDBMS (DataBricks, Snowflake, Redshift, Vertica, Clickhouse)

Good experience handling real-time, near real-time and batch data ingestions

Hands on experience on the following technologies:

Developing processes in Spark

Writing complex SQL queries f

Building ETL/data pipelines

Exposure to Kubernetes and Linux containers (i.e. Docker)

Related/complementary open source software platforms and languages (e.g. Scala, Python, Java, Linux)

Proven track record of designing effective data strategies and leveraging modern data architectures that resulted in business value

Experience building cloud-native data pipelines on either AWS, Azure or GCP, following best practices in cloud deployments

·

Strong DataOps

experience (CI/CD, Orchestration, Testing, Monitoring)

Strong experience leading and developing data engineering teams

·

Demonstrated effective interpersonal, influence, collaboration and listening skills

·

Strong stakeholder management skills

·

Excellent time management, organizational and prioritization skills with ability to balance multiple priorities.

Preferred Qualifications:

·

Experience with data tokenization and different techniques and tools i.e. DataVant, Protegrity

·

Experience with Azure Data Factory, Databricks and Snowflake

Experience with Apache Spark and related Big Data stack and technologies, PySpark Scala

·

Experience working with Apache Kafka, building appropriate producer/consumer apps

·

Experience working with Kubernetes and Docker, and knowledgeable about cloud infrastructure automation and management (e.g., Terraform)

·

Experience working in projects with agile/scrum methodologies

·

Familiarity with production quality ML and/or AI model development and deployment.

·

Healthcare industry knowledge and experience with exposure to EDI, HIPAA, HL7 and FHIR integration standards

RequirementsKubernetes, Docker

  • Operations Specialist

    1 month ago

    Delhi, India Data Engineer Academy Full time

    Job Title: Operations AnalystLocation : RemoteCompany Overview:Data Engineering Academy is a pioneering force in data engineering education, founded by FAANG data engineers with extensive industry expertise. Our platform provides comprehensive programs tailored to empower aspiring data engineers, equipping them with the skills and confidence necessary to...

  • Operations Specialist

    7 days ago

    Delhi, Delhi, India Data Engineer Academy Full time

    Job Title: Operations AnalystLocation : RemoteCompany Overview:Data Engineering Academy is a pioneering force in data engineering education, founded by FAANG data engineers with extensive industry expertise. Our platform provides comprehensive programs tailored to empower aspiring data engineers, equipping them with the skills and confidence necessary to...

  • Operations Specialist

    7 days ago

    Delhi, Delhi, India Data Engineer Academy Full time

    Job Title: Operations AnalystLocation : RemoteCompany Overview:Data Engineering Academy is a pioneering force in data engineering education, founded by FAANG data engineers with extensive industry expertise. Our platform provides comprehensive programs tailored to empower aspiring data engineers, equipping them with the skills and confidence necessary to...

  • Snowflake Data Engineer

    4 weeks ago

    delhi, India Lapis Data Analytics Full time

    Company DescriptionLapis Data Analytics is a dynamic and innovative data engineering company founded in 2020 and located in Chennai. Our primary focus is on designing, building, and maintaining robust data pipelines that convert raw data into actionable insights. With a team of seasoned experts and cutting-edge technology, we deliver customized data...

  • Snowflake Data Engineer

    7 days ago

    Delhi, Delhi, India Lapis Data Analytics Full time

    Company DescriptionLapis Data Analytics is a dynamic and innovative data engineering company founded in 2020 and located in Chennai. Our primary focus is on designing, building, and maintaining robust data pipelines that convert raw data into actionable insights. With a team of seasoned experts and cutting-edge technology, we deliver customized data...

  • Digital Engineering Sr. Engineer

    7 days ago

    Delhi, Delhi, India NTT DATA Full time

    Job DescriptionReq ID:273690NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Digital Engineering Sr. Engineer to join our team in Delhi, Delhi (IN-DL), India...

  • Digital Engineering Sr. Engineer

    7 days ago

    Delhi, Delhi, India NTT DATA Full time

    Job Description Req ID: NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Digital Engineering Sr. Engineer to join our team in Delhi, Delhi (IN-DL), India...

  • QA Engineer

    1 month ago

    delhi, India Data Unveil Full time

    Position Title: QA EngineerLevel: Mid-LevelLocation: Hyderabad, Telangana.Hire Type: Full TimeStart Date: ImmediateMinimum 3+ years of QA experience in data projects.Collaborate with the Development/Business teams to understand the requirements.Analyzing Business Requirements/cases and developing/managing the positive/negative test scenarios cases.Ability to...

  • Digital Engineering Sr. Engineer

    7 days ago

    Delhi, Delhi, India NTT DATA Services Full time

    Req ID:273690NTT DATA Services is looking for a Digital Engineering Sr. Engineer to join their team in Delhi, India.Responsibilities: Manage and maintain the Appworx/UC4 batch scheduling system Create and modify job schedules as required Monitor job execution and ensure accuracy and timeliness Troubleshoot any issues during job execution Collaborate with...

  • Tableau Developer

    3 days ago

    delhi, India The Data Family Full time

    Company DescriptionThe Data Family offers analytics and data visualization services designed to transform data into actionable insights for our clients. Through our service, customers can access expert analytics, robust data engineering, and captivating visualizations to drive better decision-making and uncover opportunities for growth.Our primary customers...

  • Software Engineer

    7 days ago

    Delhi, Delhi, India NTT DATA Full time

    Job DescriptionReq ID:283993NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Software Engineer to join our team in Delhi, Delhi (IN-DL), India (IN).Work closely with the...

  • Digital Engineering Sr. Engineer

    3 weeks ago

    New Delhi, India NTT DATA Full time

    Job DescriptionReq ID:273690NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Digital Engineering Sr. Engineer to join our team in Delhi, Delhi (IN-DL), India...

  • Digital Engineering Sr. Engineer

    3 weeks ago

    New Delhi, India NTT DATA Full time

    Job DescriptionReq ID:273690NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Digital Engineering Sr. Engineer to join our team in Delhi, Delhi (IN-DL), India...

  • Software Engineer

    7 days ago

    Delhi, Delhi, India NTT DATA Full time

    Job Description Req ID: NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Software Engineer to join our team in Delhi, Delhi (IN-DL), India (IN).Work closely with the Senior...

  • Data Engineer

    3 hours ago

    Delhi, India krtrimaIQ Cognitive Solutions Full time

    Company DescriptionkrtrimaIQ Cognitive Solutions is a technology company specializing in Cognitive Process Automation, Data Engineering, Statistical Analysis and Optimization, Data Science, Natural Language Processing, and Computer Vision. Our expertise allows us to transform organizations into intelligent enterprises.Role DescriptionThis is a full-time...

  • Junior Security Engineer

    2 days ago

    Delhi, India NTT DATA Full time

    Job DescriptionMake an impact with NTT DATAJoin a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion – it’s a place where you can grow, belong and thrive.Your day at NTT...

  • Data Engineer

    7 days ago

    Delhi, Delhi, India Aprisity Technologies Full time

    We are looking for an experienced data engineer to join our team. You will use various methods to transform raw data into useful data systems. For example, you'll create algorithms and conduct statistical analysis. Overall, you'll strive for efficiency by aligning data systems with business goals.To succeed in this data engineering position, you should have...

  • Data Engineer

    7 days ago

    Delhi, India Lingaro Full time

    Role: Data Engineer – Lead ConsultantLocation: IndiaAbout Lingaro:Lingaro Group is the end-to-end data services partner to global brands and enterprises. We lead our clients through their data journey, from strategy through development to operations and adoption, helping them to realize the full value of their data.Since 2008, Lingaro has been recognized...

  • Digital Engineering Sr. Engineer

    4 weeks ago

    new delhi, India NTT DATA Full time

    Job Description Req ID:  273690  NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Digital Engineering Sr. Engineer to join our team in Delhi, Delhi (IN-DL), India (IN)....

  • Digital Engineering Sr. Engineer

    4 weeks ago

    New Delhi, India NTT DATA Full time

    Job Description Req ID: 273690 NTT DATA Services strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Digital Engineering Sr. Engineer to join our team in Delhi, Delhi (IN-DL), India...