Data Engineer

1 day ago

Lucknow, Uttar Pradesh, India Carnation Infotech Full time ₹ 5,00,000 - ₹ 8,00,000 per year

Job Summary

As a Data Engineer , you will be responsible for designing, building, and maintaining the infrastructure and tools that enable the organization to collect, store, and analyze large volumes of data. You will work closely with data scientists, analysts, and other engineers to ensure that our data systems are reliable, efficient, and scalable.

Key Responsibilities:

Data Pipeline Development: Design, develop, and maintain scalable and reliable data pipelines to process large datasets from various sources.

Data Modeling: Collaborate with data scientists and analysts to design and implement data models that support their analytical needs.

ETL Processes: Build and optimize Extract, Transform, Load (ETL) processes to ensure data is accurately ingested and transformed.

Data Quality: Implement data quality checks and validation processes to ensure the integrity of the data.

Data Integration: Integrate data from multiple sources, including APIs, flat files, and databases, into a unified data environment.

Performance Tuning: Monitor and optimize data processing performance to meet the demands of the organization.

Documentation: Create and maintain documentation for data pipelines, data models, and other related processes.

Collaboration: Work closely with cross-functional teams to understand data needs and provide solutions that support business goals.

Compliance and Security: Ensure data handling practices comply with relevant data privacy regulations and security standards.

Qualifications:

Experience

3+ years of experience in data engineering domain.

Technical Skills:

  • Experience with ETL process (data extraction, data cleansing, data transformation, data monitoring, data modelling)
  • Experience with Microsoft Azure (Azure Data Factory, Azure Databricks, Azure Storage, Azure SQL Database, Azure Synapse)
  • Familiarity with Big Data systems (batch & stream processing, Spark, Hive)
  • Experience with two or more coding languages (including Python with pyspark)
  • Experience with SQL (Spark SQL, T-SQL),
  • Familiarity with code versioning (GIT, Azure DevOps)
  • Experience with various data formats (e.g. CSV, JSON, Parquet, Delta Lake)
  • Knowledge of distributed systems concepts and principles (consistency, availability, safety, durability, reliability, fault-tolerance)

Soft Skills:

  • Strong problem-solving and analytical skills
  • Excellent communication and collaboration skills
  • Ability to work in a fast-paced, dynamic environment

Eager to learn new things and passionate about technology

  • Data Engineer

    3 days ago

    Lucknow, Uttar Pradesh, India Carnation Infotech Full time ₹ 5,00,000 - ₹ 8,00,000 per year

    Title Data Engineer100% Work from Home / RemoteWork Timings - 1:30 PM – 10:30 PMFulltime RoleJob SummaryAs a Data Engineer , you will be responsible for designing, building, and maintaining the infrastructure and tools that enable the organization to collect, store, and analyze large volumes of data. You will work closely with data scientists, analysts,...

  • Big Data Engineer

    2 days ago

    Lucknow, Uttar Pradesh, India beBeeDataEngineer Full time ₹ 25,00,000 - ₹ 35,00,000

    Unlock a challenging and rewarding career as a data engineer with expertise in designing, developing, and maintaining large-scale data processing systems using Azure services.Key ResponsibilitiesDesign and implement scalable data pipelines to process and transform data from various sources.Develop and maintain robust data warehouses and data lakes on...

  • Data Engineering Opportunities

    4 hours ago

    Lucknow, Uttar Pradesh, India beBeeDataEngineer Full time ₹ 17,15,999 - ₹ 24,98,900

    Job Title: Data EngineerThis is an exciting opportunity to join our team as a skilled Data Engineer.You will be responsible for designing, developing, and implementing large-scale data engineering solutions to drive business growth and innovation.As a key member of our data engineering team, you will work closely with cross-functional teams to deliver...

  • Data Engineering Strategist

    13 hours ago

    Lucknow, Uttar Pradesh, India beBeeDataEngineering Full time ₹ 1,42,41,091 - ₹ 2,51,71,438

    Job Title: Data Engineering StrategistJob Overview:We are seeking an experienced data engineering leader to drive the development and implementation of high-quality data ingestion and validation processes from multiple upstream systems.Key Responsibilities:Lead the design and implementation of scalable data pipelines and validation systems across multiple...

  • Leading Data Engineer

    2 weeks ago

    Lucknow, Uttar Pradesh, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 25,00,000

    Senior Data EngineerAre you a data engineering expert looking to leverage your skills in designing, developing and optimizing scalable data pipelines? Do you have experience with Dataiku for data preparation, visualization and building end-to-end data pipelines and applications?As a Senior Data Engineer, you will lead data engineering initiatives focusing on...

  • Data Engineering Innovator

    2 days ago

    Lucknow, Uttar Pradesh, India beBeeDataEngineering Full time ₹ 15,00,000 - ₹ 25,00,000

    Data Engineering InnovatorJob Description:We are seeking a highly skilled Data Engineering Innovator to design, build, and optimize scalable data pipelines on AWS.The ideal candidate will have hands-on experience with Apache Spark for big data processing, proficiency with AWS services such as S3, Glue, Redshift, EMR, Lambda, and strong SQL skills for data...

  • Data Engineer

    3 weeks ago

    Lucknow, Uttar Pradesh, India LTIMindtree Full time

    Title : Data Engineer (Azure Synapse ) Experience: 5 to 8 years Loc: Pan India (Pune/Mumbai/Hyderabad/banagalore/Kolkata/Chennai/Noida) Interview Mode: 1 Virtual , 1 F2F Primary Skill – Experience with Azure Synapse (Synapse pipeline, IR, notebook). Secondary Skill – Databricks, PySpark, ADF Job Description : Design build and maintain scalable...

  • Data Engineer

    2 days ago

    Lucknow, Uttar Pradesh, India beBeeDataEngineering Full time ₹ 15,00,000 - ₹ 25,00,000

    Data Engineering Contract PositionWe are seeking a highly skilled Data Engineer to join our team on a contract basis. As a key member of our data engineering team, you will be responsible for designing, developing, and implementing large-scale data systems that meet the needs of our organization.Job Description:The ideal candidate will have strong Postgres...

  • Data Engineering Strategist

    2 days ago

    Lucknow, Uttar Pradesh, India beBeeData Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Senior Data Engineering LeadershipAs a high-end Senior Data Engineering Leader, you will provide strategic direction for data-driven innovation across the organization. You will oversee research, proof of concept, architecture, development, testing, deployment, and ongoing maintenance of data solutions to drive business growth.You will design and implement...

  • Data Engineering Expert

    4 days ago

    Lucknow, Uttar Pradesh, India beBeeAzure Full time ₹ 15,00,000 - ₹ 25,00,000

    Azure Data Engineer Job Opportunity">As a skilled data engineer at a leading organization, you will be responsible for migrating large-scale SQL workloads from on-premise environments to the cloud. You will design and develop end-to-end data pipelines using Azure Data Factory or Synapse Data Factory to orchestrate migration and ETL processes.">Your...