Big Data Engineer

15 hours ago

NCR, India Qcentrio Full time

Work Location : Pan India

Experience : 6+ Years

Notice Period : Immediate - 30 days

Mandatory Skills : Big Data, Python, SQL, Spark/Pyspark, AWS Cloud

JD and required Skills & Responsibilities :

  • Actively participate in all phases of the software development lifecycle, including requirements gathering, functional and technical design, development, testing, roll-out, and support.

  • Solve complex business problems by utilizing a disciplined development methodology.

  • Produce scalable, flexible, efficient, and supportable solutions using appropriate technologies.

  • Analyse the source and target system data. Map the transformation that meets the requirements.

  • Interact with the client and onsite coordinators during different phases of a project.

  • Design and implement product features in collaboration with business and Technology stakeholders.

  • Anticipate, identify, and solve issues concerning data management to improve data quality.

  • Clean, prepare, and optimize data at scale for ingestion and consumption.

  • Support the implementation of new data management projects and re-structure the current data architecture.

  • Implement automated workflows and routines using workflow scheduling tools.

  • Understand and use continuous integration, test-driven development, and production deployment frameworks.

  • Participate in design, code, test plans, and dataset implementation performed by other data engineers in support of maintaining data engineering standards.

  • Analyze and profile data for the purpose of designing scalable solutions.

  • Troubleshoot straightforward data issues and perform root cause analysis to proactively resolve product issues.

Required Skills :

  • 5+ years of relevant experience developing Data and analytic solutions.

  • Experience building data lake solutions leveraging one or more of the following AWS, EMR, S3, Hive & PySpark

  • Experience with relational SQL.

  • Experience with scripting languages such as Python.

  • Experience with source control tools such as GitHub and related dev process.

  • Experience with workflow scheduling tools such as Airflow.

  • In-depth knowledge of AWS Cloud (S3, EMR, Databricks)

  • Has a passion for data solutions.

  • Has a strong problem-solving and analytical mindset

  • Working experience in the design, Development, and test of data pipelines.

  • Experience working with Agile Teams.

  • Able to influence and communicate effectively, both verbally and in writing, with team members and business stakeholders

  • Able to quickly pick up new programming languages, technologies, and frameworks.

  • Bachelor's degree in computer science

  • Big Data Engineer

    7 days ago

    Delhi NCR, Delhi, India beBeeDataEngineer Full time ₹ 1,50,000 - ₹ 28,00,000

    We are seeking a proactive Big Data Engineer to design, implement and optimize data processing pipelines on the Databricks platform. As a key member of our team, you will be responsible for architecting scalable data solutions, collaborating with cross-functional teams to migrate existing solutions and ensuring the reliability and performance of our data...

  • Big Data Engineer

    2 weeks ago

    Delhi, NCR, India Qcentrio Full time US$ 1,50,000 - US$ 2,00,000 per year

    Work Location : Pan India Experience : 6+ Years Notice Period : Immediate - 30 days Mandatory Skills : Big Data, Python, SQL, Spark/Pyspark, AWS Cloud JD and required Skills & Responsibilities : Actively participate in all phases of the software development lifecycle, including requirements gathering, functional and technical design, development,...

  • Big Data Engineer

    15 hours ago

    NCR, India Qcentrio Full time

    We are seeking an experienced and driven Data Engineer with 5+ years of hands-on experience in building scalable data infrastructure and systems. You will play a key role in designing and developing robust, high-performance ETL pipelines and managing large-scale datasets to support critical business functions. This role requires deep technical expertise,...

  • Senior Data Engineer

    15 hours ago

    NCR, India MchooversPacific Data Integrators Full time

    Job Title: Senior Data Engineer (IDMC-CDI)Primary Skills: SQL, IDMC (CDI), ETL ConceptsMode of Work: Remote/WFHExperience: 5-10 Years About Us We are PDI (Pacific Data Integrators), a leading provider of Data Management, Analytics, and Cloud solutions. As an Informatica Platinum Partner and a reseller of various market-leading products, we help enterprises...

  • Big Data Engineer

    1 week ago

    Pune, Delhi NCR, Bengaluru / Bangalore, India beBeeDataEngineer Full time ₹ 1,04,000 - ₹ 1,30,878

    Data Engineering RoleMission Statement:Develop high-quality data solutions to support business growth and drive decision-making.Key Responsibilities:Data Ingestion and Processing: Design, develop, and deploy scalable data ingestion frameworks, real-time processing solutions, and data processing and transformation frameworks using open-source tools and data...

  • Big Data Engineer

    2 weeks ago

    Delhi, NCR, India Qcentrio Full time US$ 90,000 - US$ 1,20,000 per year

    We are seeking an experienced and driven Data Engineer with 5+ years of hands-on experience in building scalable data infrastructure and systems. You will play a key role in designing and developing robust, high-performance ETL pipelines and managing large-scale datasets to support critical business functions. This role requires deep technical expertise,...

  • Senior Big Data Engineer

    1 week ago

    Delhi NCR, Chennai, Bengaluru / Bangalore, India beBeeData Full time ₹ 15,00,000 - ₹ 28,00,000

    PySpark Developer RoleWe are seeking a highly skilled and experienced PySpark developer to join our team.As a key member of our analytics team, you will be responsible for designing, building, and optimizing large-scale data pipelines on the Databricks Unified Analytics Platform using PySpark and other big data frameworks.Develop and manage scalable ETL/ELT...

  • Senior Data Engineer

    2 weeks ago

    Delhi, NCR, India MchooversPacific Data Integrators Full time US$ 1,00,000 - US$ 1,50,000 per year

    Job Title: Senior Data Engineer (IDMC-CDI)Primary Skills: SQL, IDMC (CDI), ETL ConceptsMode of Work: Remote/WFHExperience: 5-10 YearsAbout UsWe are PDI (Pacific Data Integrators), a leading provider of Data Management, Analytics, and Cloud solutions. As an Informatica Platinum Partner and a reseller of various market-leading products, we help enterprises...

  • Azure & Big Data - Lead

    2 weeks ago

    Delhi, NCR, Pune, India Iris Software Full time ₹ 1,04,000 - ₹ 1,30,878 per year

    Job description:Iris Software has been a trusted software engineering partner to several Fortune 500 companies for over three decades. We help clients realize the full potential of technology-enabled transformation by bringing together a unique blend of domain knowledge, best-of-breed technologies, and experience executing essential and critical application...

  • Freelancer Data Engineer

    15 hours ago

    NCR, India Khaitan Education Centre Full time

    Working hours : 5-6 hours per week @Rs.400/500 per hour About the Role We are seeking a highly skilled and motivated Data Engineer to join our team for a Data Warehouse (DWH) project. The successful candidate will be responsible for designing, developing, and maintaining data pipelines and workflows to support the data warehouse's operations, ensuring data...