Datafoundry - Data Engineer - Azure Databricks

7 days ago


VishakhapatnamVizagBangaloreHyderabad, IN Datafoundry Full time

About Datafoundry AI :

Datafoundry AI is a passionate team of data scientists, product and ML engineers, and domain experts dedicated to accelerating drug discovery through the power of AI.

We are on a mission to deliver safer and more effective drugs, vaccines, and other healthcare products to patients faster. We are currently seeking a talented and results-oriented Data Engineer to join our growing team.

You will play a key role in building and maintaining the data infrastructure that supports our cutting-edge drug discovery platform.

About the Role :

This is an exciting opportunity to work on a mission-critical project with a real-world impact on healthcare. You will leverage your expertise in data engineering to design, develop, and implement scalable data pipelines for data ingestion, transformation, and loading (ETL) of massive datasets (in TBs).

Responsibilities :

- Design and develop efficient ETL pipelines using SQL (PostgreSQL or equivalent) and PL/SQL.

- Utilize Azure Databricks for large-scale data processing and transformation tasks.

- Migrate data from various sources to our data lake and data warehouse.

- Ensure data quality, consistency, and integrity throughout the data lifecycle.

- Collaborate with data scientists and analysts to understand data needs and design solutions.

- Develop and maintain data pipelines that are scalable, maintainable, and secure.

- Automate data processing tasks using scripting languages (e.g., Python, Bash).

- Monitor and optimize data pipelines for performance and efficiency.

- Stay up-to-date with the latest advancements in data engineering technologies and tools.

Qualifications :

- Bachelor's degree in Computer Science, Information Technology, or a related field (or equivalent experience).

- 3+ years of experience in data engineering with a strong focus on ETL processes.

- Proficiency in SQL (PostgreSQL or equivalent) and PL/SQL for data manipulation and querying.

- Experience with cloud-based data processing platforms (e.g., Azure Databricks) is a plus.

- Experience with data migration methodologies and tools is a plus.

(ref:hirist.tech)

  • Vishakhapatnam/Vizag/Bangalore/Hyderabad, IN Datafoundry Full time

    About Datafoundry AI :At Datafoundry AI, we're a committed team of data scientists, life sciences experts, and technology enthusiasts. Our primary goal is to optimize DL and ML models for maximum impact, offering comprehensive support across all facets of the data-driven business. Who are we looking for : We're in search of a dynamic, accountable...


  • Hyderabad/Vishakhapatnam/Vizag, IN Datafoundry Full time

    About Datafoundry AI : Datafoundry AI is a passionate team of data scientists, product and ML engineers, and domain experts dedicated to accelerating drug discovery through the power of AI. We are on a mission to deliver safer and more effective drugs, vaccines, and other healthcare products to patients faster.We are currently seeking talented and innovative...


  • Pune/Bangalore/Hyderabad, IN Advitha Tech Solutions Full time

    Job Description :- Should have 10+ years of experience, minimum 8+ in Data engineering and Analytics with Microsoft cloud technologies.- Expertise to Implementing Data warehousing Solutions. - experience as Data Engineer in Azure Data Bricks with design and solutioning.- Strong Programming experience in Python, SQL- Must have Strong technical experience in...


  • Pune/Bangalore/Hyderabad, IN Advitha Tech Solutions Full time

    Job Description :- Should have 10+ years of experience, minimum 8+ in Data engineering and Analytics with Microsoft cloud technologies.- Expertise to Implementing Data warehousing Solutions. - experience as Data Engineer in Azure Data Bricks with design and solutioning.- Strong Programming experience in Python, SQL- Must have Strong technical experience in...


  • Hyderabad/Pune/Bangalore, IN manogna it services private limited Full time

    Job Description :- Should have 10+ years of experience, minimum 8+ in Data engineering and Analytics with Microsoft cloud technologies.- Expertise to Implementing Data warehousing Solutions. - Experience as Data Engineer in Azure Data Bricks with design and solutioning.- Strong Programming experience in Python, SQL- Must have Strong technical experience in...


  • Hyderabad/Pune/Bangalore, IN manogna it services private limited Full time

    Job Description :- Should have 10+ years of experience, minimum 8+ in Data engineering and Analytics with Microsoft cloud technologies.- Expertise to Implementing Data warehousing Solutions. - Experience as Data Engineer in Azure Data Bricks with design and solutioning.- Strong Programming experience in Python, SQL- Must have Strong technical experience in...

  • Data Engineer

    3 weeks ago


    Pune/Hyderabad, IN EDGESOFT Full time

    Job Description :The ideal candidate should have a robust understanding and hands-on expertise in PySpark and various components within DataBricks. As a crucial member of our data team, you will play a pivotal role in developing, optimizing, and maintaining our data infrastructure, ensuring seamless and efficient data processing.Responsibilities :- Design,...

  • Data Engineer

    3 weeks ago


    Pune/Hyderabad, IN EDGESOFT Full time

    Job Description :The ideal candidate should have a robust understanding and hands-on expertise in PySpark and various components within DataBricks. As a crucial member of our data team, you will play a pivotal role in developing, optimizing, and maintaining our data infrastructure, ensuring seamless and efficient data processing.Responsibilities :- Design,...

  • Data Engineer

    3 weeks ago


    Pune/Hyderabad/Remote, IN HARP Technologies and Services Full time

    Job Location : Pune & Hyderabad (Initial 1 -2 months hybrid and then complete remote job. Client will offer accommodation(stay), food & travel exp. )Exp range : 6+ years Shift timings : General 9am - 6pm IST or 10am - 7pm IST Mandatory skills : Data engineering (4+years), Databricks (3.5+ years), Pyspark (3+ years), Python (preferred) or bash, Data pipeline,...

  • Data Engineer

    2 months ago


    Pune/Hyderabad/Remote, IN HARP Technologies and Services Full time

    Job Location : Pune & Hyderabad (Initial 1 -2 months hybrid and then complete remote job. Client will offer accommodation(stay), food & travel exp. )Exp range : 6+ years Shift timings : General 9am - 6pm IST or 10am - 7pm IST Mandatory skills : Data engineering (4+years), Databricks (3.5+ years), Pyspark (3+ years), Python (preferred) or bash, Data pipeline,...

  • Azure Data Engineer

    3 weeks ago


    Pune/Bangalore/Chennai, IN Growel Softech Pvt. Ltd. Full time

    Job Title : Azure Data EngineerLocations : Pune, Bangalore, ChennaiExperience Required : 4 - 10 yearsJob Description :We are looking for an experienced Azure Data Engineer to join our dynamic team. The ideal candidate will have a strong background in data engineering with hands-on experience in designing and implementing data solutions using Microsoft Azure....

  • Senior Data Engineer

    2 months ago


    Chennai/Bangalore/Hyderabad/Pune, IN Resourcetree Full time

    Job Description :Data engineering role requires creating and managing technological infrastructure of a data platform, be in- charge / involved in architecting, building, and managing data flows / pipelines and construct data storages (noSQL, SQL), tools to work with big data (Hadoop, Kafka), and integration tools to connect sources or other databases.Role...

  • Data Engineering Lead

    1 month ago


    Coimbatore/Tamil Nadu/Kerala/Chennai/Pondicherry/Bangalore/Cochin/Kochi/Hosur, IN QBrainX Full time

    Job Title : Data Engineering Lead (With Architecture Experience) Exp level : 10+ yearsLocation : Coimbatore Position Type : Full-Time No of opening : 2- Immediate Joiner (OR) Who can join us within 15 to 20 days is preferred.Key Responsibilities : Architect and Design Data Solutions : - Collaborate with cross-functional teams to understand business...

  • Data Engineering Lead

    3 weeks ago


    Coimbatore/Tamil Nadu/Kerala/Chennai/Pondicherry/Bangalore/Cochin/Kochi/Hosur, IN QBrainX Full time

    Job Title : Data Engineering Lead (With Architecture Experience) Exp level : 10+ yearsLocation : Coimbatore Position Type : Full-Time No of opening : 2- Immediate Joiner (OR) Who can join us within 15 to 20 days is preferred.Key Responsibilities : Architect and Design Data Solutions : - Collaborate with cross-functional teams to understand business...


  • Anywhere in India/Multiple Locations/Bangalore, IN Notus Full time

    Job Title : Azure Databricks EngineerLocation : Pan IndiaJob Type : Immediate to 15-Day JoinerExperience Level : 6+ YearsJob Summary :We are seeking an experienced Azure Databricks Engineer to join our team in Pan India. The ideal candidate will have a strong background in Azure Data Engineering, with expertise in Azure Databricks, Azure Data Factory, Azure...


  • Anywhere in India/Multiple Locations/Bangalore, IN Notus Full time

    Job Title : Azure Databricks EngineerLocation : Pan IndiaJob Type : Immediate to 15-Day JoinerExperience Level : 6+ YearsJob Summary :We are seeking an experienced Azure Databricks Engineer to join our team in Pan India. The ideal candidate will have a strong background in Azure Data Engineering, with expertise in Azure Databricks, Azure Data Factory, Azure...

  • Databricks Engineer

    3 weeks ago


    Pune/Hyderabad/Indore/Ahmedabad/Gurgaon/Gurugram, IN Arting Digital Full time

    Job Title : Databricks Engineer Experience : 7+yrLocation : Hyderabad, Indore, Ahmadabad, Pune, GurgaonWork Mode : OnsiteQualification : B.TECH, M.tech, MCASkills : Databricks, Databricks Admin, Spark, Azure, Docker, Github, Jenkins and Linux/Unix etc.Requirements :- Bachelor's degree in computer science or related field. - 7+ years of IT experience,...

  • Azure Data Architect

    1 month ago


    Bangalore/Chennai/Hyderabad/Mumbai/Pune/Noida, IN ACZ Global Private Limited Full time

    Azure Data ArchitectMandatory Skills : Solution Architecture - Pyspark + Databricks + Adf + Synapse is mandatoryJob Description :We are seeking a highly skilled and experienced Azure Data Architect to join our team. As an Azure Data Architect, you will play a key role in designing and implementing data solutions on the Microsoft Azure platform. The ideal...

  • Data Engineer

    2 months ago


    Bangalore/Hyderabad/Chennai/Pune, IN Nilasu consulting Full time

    Responsibilities:- Design, develop, and deploy production-ready data pipelines using Databricks, Kafka, and Snowflake.- Collaborate with data analysts, software engineers, and stakeholders to understand data requirements and develop effective data migration strategies, API configurations, and backend architecture.- Extract, transform, and load (ETL) data...

  • Data Engineer

    3 weeks ago


    Bangalore/Hyderabad/Chennai/Pune, IN Nilasu consulting Full time

    Responsibilities:- Design, develop, and deploy production-ready data pipelines using Databricks, Kafka, and Snowflake.- Collaborate with data analysts, software engineers, and stakeholders to understand data requirements and develop effective data migration strategies, API configurations, and backend architecture.- Extract, transform, and load (ETL) data...