QBrainX - Data Engineering Lead - Python/PySpark

2 months ago


Chennai, India QBrainX Full time

Job title - Data Engineering Lead

Exp - 6 to 8 years

Job Location : Coimbatore / Hyderabad (Work from Office)

Do you love building the infrastructure that fuels data-driven decision making? Are you passionate about wrangling raw data into a usable format? If so, then we want to hear from you

We are searching for a Data Engineering Lead to join our growing team. In this role, you will play a key part in designing, developing, and maintaining our data pipelines. You will work closely with data scientists, analysts, and business stakeholders to ensure our data infrastructure meets the ever-evolving needs of the :

- Design, develop, and maintain scalable data pipelines using technologies like Python and SQL.

- Develop and implement data quality checks and monitoring processes to ensure data integrity.

- Build and automate data ingestion processes from various sources (databases, APIs, etc.) to Snowflake.

- Utilize cloud platforms (AWS, Azure, GCP) to build and manage data infrastructure.

- Leverage Infrastructure as Code (IaC) tools like Terraform to provision and manage cloud resources.

- Collaborate with data scientists and analysts to understand data requirements and translate them into technical solutions.

- Version control code using Git and collaborate with the team through code reviews.

- Document data pipelines and processes for future reference and knowledge sharing.

Qualifications :

- 6 years of experience in data engineering or a related field.

- Proven experience in designing, developing, and deploying data pipelines.

- Hands on experience with Snowflake, DataBricks, Python or Pyspark and SQL.

- Should have experience working in Data Build Tool (DBT) with snowflake is required

- Experience with data warehousing and data modeling concepts.

- Experience working with cloud platforms (AWS, Azure, GCP) is a plus.

- Working knowledge of Terraform and Infrastructure as Code (IaC) principles.

- Experience with Git for version control.

- Excellent communication and collaboration skills.

- Ability to work independently and as part of a team.

- Should have experience in leading the data engineering team.

- Knowledge and Experience toward Architectural and design is added advantage.

- Strong problem-solving and analytical skills.

Bonus Points :

- Experience with data orchestration tools (Airflow, Luigi, etc.).

- Experience with data visualization tools (Tableau, Power BI, etc.)

(ref:hirist.tech)

  • Anywhere in India/Multiple Locations/Hyderabad/Chennai/Kerala/Andhra Pradesh/Coimbatore QBrainX Full time

    Data Engineering Lead PositionThe ideal candidate for this role will have extensive experience in designing, developing, and maintaining large-scale data pipelines using Python and SQL. Proficiency in cloud platforms such as AWS, Azure, and GCP is a plus, as well as experience with Terraform and Infrastructure as Code (IaC) principles.Key Responsibilities :-...


  • Chennai, Tamil Nadu, India QBrainX Full time

    Job Title - Senior Data Engineering LeadWe are seeking a highly skilled Data Engineering Lead to join our team at QBrainX. As a key member of our data engineering team, you will play a crucial role in designing, developing, and maintaining our data pipelines.Estimated Salary: $120,000 - $180,000 per yearResponsibilities:Design, develop, and maintain scalable...


  • Anywhere in India/Multiple Locations/Hyderabad/Chennai/Kerala/Andhra Pradesh/Coimbatore QBrainX Full time

    Job Title: Data Engineering LeadWe are seeking an experienced Data Engineering Lead to join our team at QBrainX. This role involves designing, developing, and maintaining scalable data pipelines using technologies like Python and SQL.About the Role:In this position, you will play a key part in ensuring our data infrastructure meets the ever-evolving needs of...


  • Anywhere in India/Multiple Locations/Hyderabad/Chennai/Kerala/Andhra Pradesh/Coimbatore QBrainX Full time

    Job title - Data Engineering Lead Exp - 6 to 8 years Job Location : Coimbatore / Hyderabad (Work from Office) Do you love building the infrastructure that fuels data-driven decision making? Are you passionate about wrangling raw data into a usable format? If so, then we want to hear from you!We are searching for a Data Engineering Lead to join our growing...


  • Chennai, Tamil Nadu, India QBrainX Full time

    About the RoleQBrainX is seeking a highly skilled AI Solutions Strategist to lead our efforts in developing and deploying AI systems. As a key member of our team, you will be responsible for defining and implementing our AI architecture strategy, ensuring it aligns with our business objectives.Key ResponsibilitiesDevelop and implement AI architecture...

  • Data Engineer

    1 month ago


    Bangalore/Chennai, India TekPillar Full time

    Job Title : Data EngineerExperience : 3-8 YearsLocation : Chennai / BangaloreJob Type : Full-timeRequire Skills : Data Engineer, AWS, Airflow, PySpark, Hadoop, Big data, Glue, Athena, Lambda, Python Job Description : We are seeking a highly skilled and experienced Data Engineer with 3 to 8 years of experience. - Strong experience with AWS services:...

  • Data Architect Lead

    3 weeks ago


    Chennai, Tamil Nadu, India QBrainX Full time

    About This Role:As a Data Engineering Lead at QBrainX, you will be responsible for designing and implementing scalable and efficient data solutions. Your expertise in Snowflake, Azure, and Databricks technologies will be crucial in building and optimizing data warehouses. You will collaborate with cross-functional teams to understand business requirements...


  • Chennai, India RiskInsight Consulting Pvt Ltd Full time

    Responsibilities Design, develop, and deploy scalable data processing applications using Python and PySpark. Collaborate with data scientists and analysts to understand requirements and translate them into technical solutions. Write efficient and optimized code to process and analyze large volumes of data. Implement data ingestion processes from various data...


  • Chennai, Tamil Nadu, India QBrainX Full time

    Job Title : Cloud Data Solutions ArchitectLocation : Coimbatore (Work From Office)Salary Range : ₹2500000 - ₹4000000 per annumAbout the RoleWe are seeking a highly skilled Cloud Data Solutions Architect to join our team at QBrainX. As a key member of our data engineering team, you will be responsible for designing and implementing scalable and efficient...

  • Big Data Engineer

    3 weeks ago


    Chennai, India Changeleaders Full time

    Job Summary : We are seeking a highly skilled and motivated Big Data Engineer to join our dynamic team. The ideal candidate will have hands-on experience in designing, developing, and optimizing scalable big data pipelines and solutions. This role requires proficiency in advanced big data tools, technologies, and programming languages.Key Responsibilities :...

  • Data Engineer

    3 months ago


    Chennai, India HNM Solutions Full time

    Role : Data EngineerLocation : Chennai (Candidate will work from Client office)Experience : 7 years( Relevant - 4.5 year Mandatory)NOTE : Only immediate joinersCandidate current location should be Chennai only....Description : We are seeking a Data Engineer who will build and maintain data pipelines across business areas such as subscriptions, video,...

  • Data Engineer

    1 week ago


    Chennai, India HNM Solutions Full time

    Role : Data Engineer Location : Chennai (Candidate will work from Client office) Experience : 7 years( Relevant - 4.5 year Mandatory) NOTE : Only immediate joiners Candidate current location should be Chennai only.... Description : We are seeking a Data Engineer who will build and maintain data pipelines across business areas such as subscriptions,...

  • TechStar Group

    2 months ago


    Chennai, India Techstar Group Full time

    Bachelor's in engineering / Master's degree in Computer Science, Information Systems and preferred to have domain knowledge in MortgageJob Description :- Minimum of 7 of IT experience in data engineering.- Strong technical expertise with data models in PostgreSQL/Snowflake.- Extensive experience in Pyspark and performance tuning.- Experience with...

  • TechStar Group

    1 week ago


    Chennai, India Techstar Group Full time

    Bachelor's in engineering / Master's degree in Computer Science, Information Systems and preferred to have domain knowledge in Mortgage Job Description :- Minimum of 7 of IT experience in data engineering.- Strong technical expertise with data models in PostgreSQL/Snowflake.- Extensive experience in Pyspark and performance tuning.- Experience with...

  • Colan Infotech

    2 months ago


    Chennai, India Colan Infotech Pvt Ltd Full time

    Role : Azure Data EngineerExperience : 5 Years Skill Set : Pyspark / Scala Spark, Data Factory, Databricks, Python, SQL. - Must have cloud knowledge in AzureRoles and Responsibilities :- Should have programming skills with the ability to write optimized and reusable high-quality code.- Design, develop and maintain scalable data pipelines using Pyspark /...

  • Colan Infotech

    1 week ago


    Chennai, India Colan Infotech Pvt Ltd Full time

    Role : Azure Data EngineerExperience : 5 Years Skill Set : Pyspark / Scala Spark, Data Factory, Databricks, Python, SQL. - Must have cloud knowledge in AzureRoles and Responsibilities :- Should have programming skills with the ability to write optimized and reusable high-quality code.- Design, develop and maintain scalable data pipelines using Pyspark /...


  • Hyderabad/Bangalore/Chennai/Vishakhapatnam/Vizag, India Mouri Tech (P) Ltd Full time

    Senior Cloud Data Engineer (AWS, Python, Pyspark) Job Title : Senior AWS Cloud Data EngineerWork Location : Hyderabad/Kolhapur/Bangalore/Chennai Experience : 6 to 9 yearsRequired Skills : Python, Pyspark, AWS services such as DMS, SAM, Glue, Redshift, Lamda S3 with Apache Airflow, CI/CD, : - Minimum of 6-9 Experience with the core AWS Services and setting...

  • Pyspark Developer

    3 weeks ago


    chennai, India Tata Consultancy Services Full time

    Greetings from TCS!! TCS is hiring for Data Engineer (Pyspark Developer) Required Skill Set: Pyspark, Python (Pandas and Numpy), Azure Synapse/ADF/Databricks , SQL and relational databases, SparkSQL, Spark Scripting, UNIX Shell Scripting, ETL, Data Warehousing, DevOps (CI/CD) Desired Experience Range: 6 to 10 Years Job Location: Chennai / Pune Must...

  • Pyspark Developer

    3 weeks ago


    Chennai, India Tata Consultancy Services Full time

    Greetings from TCS!!TCS is hiring for Data Engineer (Pyspark Developer)Required Skill Set: Pyspark, Python (Pandas and Numpy), Azure Synapse/ADF/Databricks , SQL and relational databases, SparkSQL, Spark Scripting, UNIX Shell Scripting, ETL, Data Warehousing, DevOps (CI/CD)Desired Experience Range: 6 to 10 YearsJob Location: Chennai / PuneMust Have:Data...

  • Pyspark Developer

    3 weeks ago


    chennai, India Tata Consultancy Services Full time

    Greetings from TCS!!TCS is hiring for Data Engineer (Pyspark Developer)Required Skill Set: Pyspark, Python (Pandas and Numpy), Azure Synapse/ADF/Databricks , SQL and relational databases, SparkSQL, Spark Scripting, UNIX Shell Scripting, ETL, Data Warehousing, DevOps (CI/CD)Desired Experience Range: 6 to 10 YearsJob Location: Chennai / PuneMust Have:Data...