Big Data Engineer

1 week ago

Any Location, IN Programming.com Full time

Job Title : Big Data Engineer

Location : Remote

About Us :

We are a leading specializing in software development . Our team is dedicated to leveraging cutting-edge technologies to deliver scalable and efficient solutions to our clients globally.

Role Overview :

We are seeking a highly skilled Big Data Engineer with a strong background in designing and developing robust solutions using big data technologies. The ideal candidate will be proficient in Python, PySpark, and have hands-on experience with cloud platforms such as Azure, AWS, or GCP.


This role requires expertise in data manipulation, querying across various database types, and a strong focus on building scalable ingestion pipelines.

Key Responsibilities :

- Design, develop, and deploy scalable solutions using big data technologies.

- Utilize Python and PySpark for efficient data processing and analysis.

- Work with cloud platforms (Azure, AWS, GCP) to deploy and optimize data solutions.

- Query and manipulate data from various databases including relational and big data platforms.

- Write effective and maintainable unit and integration tests for ingestion pipelines.

- Implement static analysis, code quality tools, and CI/CD pipelines to automate software delivery.

- Collaborate effectively with cross-functional teams to deliver high-quality solutions in a fast-paced, dynamic environment.

Provide technical leadership, mentorship, and guidance to junior team members.

Requirements :

- Bachelor's degree in Computer Science, Engineering, or related field (Master's preferred).

- 6+ years of experience in design and development with a focus on big data technologies.

- Proficiency in Python and PySpark for data processing and analysis.

- 2+ years of hands-on development experience with cloud technologies (Azure, AWS, GCP).

- Strong experience in querying and manipulating data from various DB (relational and big data).

- Demonstrated expertise in writing unit and integration tests for ingestion pipelines.

- Familiarity with static analysis tools, code quality tools, and building CI/CD pipelines.

- Excellent communication skills and ability to work collaboratively in a team environment.

- Proven ability to thrive in high-traffic and large-scale software product environments.

Preferred Qualifications :

- Experience with streaming data technologies (e.g., Kafka, Apache Spark Streaming).

- Knowledge of containerization and orchestration tools (e.g., Docker, Kubernetes).

- Certification in relevant technologies (e.g., AWS Certified Big Data Specialty, Azure Data Engineer Associate).

Benefits :

- Competitive salary and performance-based incentives.

- Comprehensive health benefits package.

- Flexible work hours and remote work options.

- Professional development opportunities and certifications.

- Vibrant company culture with team-building activities and social events.

(ref:hirist.tech)

  • Big Data Engineer

    4 weeks ago

    Any Location/Bangalore, IN Careerfit.ai Full time

    Responsibilities :Solution Design :- Design, develop, and maintain scalable big data solutions and data pipelines for efficient data ingestion, processing, storage, and analysis.- Collaborate with data scientists, analysts, and business stakeholders to identify data requirements, use cases, and business objectives.- Translate business needs into technical...

  • Big Data Engineer

    2 months ago

    Any Location/Bangalore, IN Careerfit.ai Full time

    Responsibilities :Solution Design :- Design, develop, and maintain scalable big data solutions and data pipelines for efficient data ingestion, processing, storage, and analysis.- Collaborate with data scientists, analysts, and business stakeholders to identify data requirements, use cases, and business objectives.- Translate business needs into technical...

  • Big Data Solutions Engineer

    4 weeks ago

    Bangalore/Any Location, IN Gethyr Full time

    Our company is seeking a talented Big Data Solutions Engineer to join our team.- The ideal candidate will be responsible for designing, developing, and implementing our proprietary Bigdata-powered Fin Crime solution for our clients.- The candidate will work closely with the software development team, data scientists, and product managers to understand...

  • Big Data Solutions Engineer

    4 weeks ago

    Bangalore/Any Location, IN Gethyr Full time

    Our company is seeking a talented Big Data Solutions Engineer to join our team.- The ideal candidate will be responsible for designing, developing, and implementing our proprietary Bigdata-powered Fin Crime solution for our clients.- The candidate will work closely with the software development team, data scientists, and product managers to understand...

  • Senior Machine Learning Engineer

    4 weeks ago

    Any Location, IN Tekvaly Full time

    Job Description :As a Senior Machine Learning Engineer, you will be responsible for leading the development and deployment of machine learning models. You will work closely with data scientists, data engineers, and other stakeholders to understand business requirements and translate them into scalable ML solutions.Key Responsibilities :Model Development and...

  • Senior Machine Learning Engineer

    4 weeks ago

    Any Location, IN Tekvaly Full time

    Job Description :As a Senior Machine Learning Engineer, you will be responsible for leading the development and deployment of machine learning models. You will work closely with data scientists, data engineers, and other stakeholders to understand business requirements and translate them into scalable ML solutions.Key Responsibilities :Model Development and...

  • Senior Data Engineer

    3 weeks ago

    Any Location, IN UNICO CONNECT PRIVATE LIMITED Full time

    Job Description :Key Responsibilities :- Understand the factories , manufacturing process , data availability and avenues for improvement- Brainstorm , together with engineering, manufacturing and quality problems that can be solved using the acquired data in the data lake platform.- Define what data is required to create a solution and work with...

  • Senior Machine Learning Engineer

    2 months ago

    Any Location, IN K2 Data & Analytics Full time

    Job Description : We are on the lookout for an enthusiastic ML Engineer with over five years of experience to be part of our team, which is focused on developing cutting-edge Video Ad Data technologies. The perfect fit for this role would be someone with a solid grasp of machine learning algorithms and a background in designing and executing large-scale...

  • Senior Machine Learning Engineer

    4 weeks ago

    Any Location, IN K2 Data & Analytics Full time

    Job Description : We are on the lookout for an enthusiastic ML Engineer with over five years of experience to be part of our team, which is focused on developing cutting-edge Video Ad Data technologies. The perfect fit for this role would be someone with a solid grasp of machine learning algorithms and a background in designing and executing large-scale...

  • Data Engineer

    4 weeks ago

    Bangalore/Any Location, IN INSIGHT GLOBAL SOLUTIONS Full time

    Data Engineer Must Haves :- Bachelors Degree in science, technology, or another related field.- 3+ years of data engineering experience.- 3+ years of experience with Cloud platforms like GCP, Azure, Airflow.- 4+ years of data warehouse, relational database, and SQL experience.- 4+ years of supporting ETL process and job scheduling experience.- 3+ years of...

  • Data Engineer

    2 months ago

    Bangalore/Any Location, IN INSIGHT GLOBAL SOLUTIONS Full time

    Data Engineer Must Haves :- Bachelors Degree in science, technology, or another related field.- 3+ years of data engineering experience.- 3+ years of experience with Cloud platforms like GCP, Azure, Airflow.- 4+ years of data warehouse, relational database, and SQL experience.- 4+ years of supporting ETL process and job scheduling experience.- 3+ years of...

  • Senior Data Engineer

    4 weeks ago

    Any Location, IN Coders Brain Technology Pvt. Ltd. Full time

    Position Name : Sr Data EngineerExperience Required : 8+yearsSalary : As per the Market Standard or Else whatever budget you haveNotice period : Immediate joiner Job Type : Full timeLocation : RemoteJob Description :Solution Architect (Databricks) :Candidate having 8+ years of total experience in Data engineering (development/design), in which at least 5+...

  • Senior Data Engineer

    4 weeks ago

    Any Location, IN Coders Brain Technology Pvt. Ltd. Full time

    Position Name : Sr Data EngineerExperience Required : 8+yearsSalary : As per the Market Standard or Else whatever budget you haveNotice period : Immediate joiner Job Type : Full timeLocation : RemoteJob Description :Solution Architect (Databricks) :Candidate having 8+ years of total experience in Data engineering (development/design), in which at least 5+...

  • Big Data DevOps Engineer

    4 weeks ago

    Any Location, IN Freelancer Recruiter Full time

    Job Requirements :- 5+ years of DevOps experience managing the Big Data application stack including HDFS, YARN, Spark, Hive and Hbase- A deeper understanding of all the configurations required for installing and maintaining the infrastructure in the long run- Experience setting up high availability, configuring resource allocation, setting up capacity...

  • Big Data DevOps Engineer

    2 months ago

    Any Location, IN Freelancer Recruiter Full time

    Job Requirements :- 5+ years of DevOps experience managing the Big Data application stack including HDFS, YARN, Spark, Hive and Hbase- A deeper understanding of all the configurations required for installing and maintaining the infrastructure in the long run- Experience setting up high availability, configuring resource allocation, setting up capacity...

  • Senior Data Engineer

    2 months ago

    Bangalore/Any Location, IN Randstad Full time

    Job Description:We are looking for a highly skilled Senior Data Engineer with over 5 years of experience in the data engineering field. The ideal candidate will have strong expertise in PySpark, SQL databases, and Big Data programming for data transformation and aggregations. This role offers an exciting opportunity to work on cutting-edge projects,...

  • Senior Data Engineer

    4 weeks ago

    Bangalore/Any Location, IN Randstad Full time

    Job Description:We are looking for a highly skilled Senior Data Engineer with over 5 years of experience in the data engineering field. The ideal candidate will have strong expertise in PySpark, SQL databases, and Big Data programming for data transformation and aggregations. This role offers an exciting opportunity to work on cutting-edge projects,...

  • Azure Data Engineer

    2 months ago

    Any Location, IN Visionyle Full time

    Position Name: Azure Data EngineerPosition Mode: PermanentExperience: 6 YearsJoining Location: RemoteNotice: ImmediateMode of Interview : 2-3 roundsSUMMARY:Azure Data Engineer should have experience on Azure data lake, Azure SQL database, Databricks, Azure Data Factory, and Azure SQL data warehouse and controlling and granting database access and migrating o...

  • Azure Data Engineer

    4 weeks ago

    Any Location, IN Visionyle Full time

    Position Name: Azure Data EngineerPosition Mode: PermanentExperience: 6 YearsJoining Location: RemoteNotice: ImmediateMode of Interview : 2-3 roundsSUMMARY:Azure Data Engineer should have experience on Azure data lake, Azure SQL database, Databricks, Azure Data Factory, and Azure SQL data warehouse and controlling and granting database access and migrating o...

  • Victrix Systems and Labs

    4 weeks ago

    Any Location, IN Victrix Systems & Labs Full time

    Job Title : GCP Cloud Data EngineerLocation : RemoteExperience : 2 to 7 yearsJob Description :We are seeking a talented Cloud Data Engineer to join our team on a remote basis.The ideal candidate will have a strong background in cloud migration, building ETL pipelines, data integrations, and the development of Operational Data Stores (ODS) and Data Warehouses...