Excelra - Lead Data Engineer - Python/Spark

2 weeks ago

Anywhere in IndiaMultiple LocationsAny LocationDelhi NCRMumbaiBangaloreHyderabadKolkataChenn Excelra Knowledge Solutions Full time

Job Purpose :


This role contributes to the construction of the development data fabric and data strategy. This role will interact with architects, engineers, data modelers, product owners as well as other team members in Clinical Solutions and R&D.

This role will actively participate in creating technical solutions, designs, implementations & participate in the relentless improvement of R&D Tech systems in alignment with agile and DevOps principles.

The Data Engineer demonstrates both depth and breadth across key data engineering competencies e.g. Software Development, Testing, DevOps, Data Science/Analytics, and cloud. Can collaborate with experts from other subject domains.

Primary responsibilities include using Azure cloud services and Client data platform tools to ingest, egress, and transform data from multiple sources.

In addition, the role will demonstrate core engineering knowledge/experience of industry technologies, practices, and frameworks such as data fabric and scaling data platforms, containerization, cloud-based platforms, data analytics, machine learning, and data streaming.

Examples of technologies include :

Java/C#/Python, Denodo, Snowflake, GIT, Azure DevOps, Data Bricks, Presto, Spark, Azure Data Factory, ADLS V2, Kafka, Selenium, JUnit/N Unit, Safe, Kanban, Docker, AI/ML, Azure/GCP Cloud Architecture including networking principles and scaling applications.

The Data Engineer, Clinical Solutions role is a senior technical role and will provide you the opportunity to lead key activities to progress your career.

These responsibilities include the following :

- Working with other teams that are defining DevOps and data platform practices to meet the requirements of clinical solutions.

- Supporting engineering teams in the adoption and creation of data fabric best practices.

- Conducting POCs of new technologies and helping to embed them in product teams

- Being part of a cutting-edge team creating the Development Data Fabric

- Ensures that technical delivery is fully compliant with Client Security, Quality and Regulatory standards.

- Ensures use of relevant R&D Tech / central services and collaborating with service partners in identification and delivery of service improvements

- Maintains best practices for engineering and architecture on our Confluence site. This requires hands on experience with cutting edge technology.

- Pro-actively engages in experimentation and innovation to drive relentless improvement.

- Provides leadership, technical direction and client expertise to architecture and engineering teams composed of client FTEs, strategic partners and software vendors.

We are looking for professionals with these required skills to achieve our goals:

- Total 11+ years of experience and proficient with at least 3 of the below skills and can demonstrate knowledge and value with relevant experience in all the following competencies:

- Must have experience in Spark, Python and Databricks

- Software development, architecture design & technology platforms/frameworks

- Data Platforms and Domain-driven design

- Agile, DevOps & Automation (of testing, build, deployment, CI/CD, etc.)

- Data science (e.g. AI/ML), data analytics & data quality/integrity

- Testing strategies & frameworks

Role requires :

- Demonstrated skill in delivering high-quality engineered data products

- Knowledge of industry standards and technology platforms aligned to Client and R&D roadmaps

- Excellent communication, negotiation, influencing and stakeholder management skills

- Customer focus and excellent problem-solving skills

- Computer Science or related bachelor's degree - MS in Computer Science is preferred

- Familiarity and use of various open-source ecosystems including JavaScript, Bigdata, java, python etc.

- Good understanding of various software paradigms: domain-driven, procedural, data-driven, object-oriented, functional

- Familiar with .NET Core (C#), Java, Python

- Demonstrable knowledge depth in more than one area of software engineering and technology

Preferred Qualifications :

If you have the following characteristics, it would be a plus :

- Experience in agile software development and DevOps, relevant technology platforms (e.g., Kubernetes) and frameworks (e.g. Docker) including cloud technologies & data structures (i.e. information management), data models or relational database design

- Subject matter expertise in clinical development

- R&D Tech requires Engineers with understanding of the relevant technical and scientific domains. Able to deliver continuous change to meet rapidly evolving R&D strategy and ambition.

- Experience with agile development methods, with security strategies and best practices, data integration mechanisms, architectural design tools, delivering and integrating COTS applications, areas of Service Oriented Architecture (SOA), Application Integration, Business Process Management and Data Quality.

- Experience in applying AI/ML, data curation, virtualization, predictive modelling, workflow, and advanced visualization techniques to enable decision support across multiple products and assets to drive results across R&D business operations.

(ref:hirist.tech)

  • Excelra - Senior Data Scientist - Artificial Intelligence/Machine Learning/NLP

    2 weeks ago

    Anywhere in India/Multiple Locations Excelra Knowledge Solutions Full time

    Job Title : Data ScientistLocation : Remote Company : ExcelraExcelra, a leader in biopharma data and analytics, uniquely blends scientific understanding with technological expertise. This dual mastery empowers innovative drug discovery and development, transforming research data into actionable insights for groundbreaking discovery.Job Description :- As a...

  • Excelra - Golang Backend Developer

    2 weeks ago

    Anywhere in India/Multiple Locations Excelra Knowledge Solutions Full time

    About Team : Excelra is hiring for a new path in health research with its clinical trial automation platform, contributing to the ongoing digitalization of health innovation. We emphasize a collaborative team environment, where hard work and dedication are acknowledged through opportunities for advancement into senior roles.Our focus is on nurturing...

  • Excelra - Golang Backend Developer

    2 weeks ago

    Anywhere in India/Multiple Locations, IN Excelra Knowledge Solutions Full time

    About Team : Excelra is hiring for a new path in health research with its clinical trial automation platform, contributing to the ongoing digitalization of health innovation. We emphasize a collaborative team environment, where hard work and dedication are acknowledged through opportunities for advancement into senior roles.Our focus is on nurturing employee...

  • Data Engineer

    2 weeks ago

    Anywhere in India/Multiple Locations Paralleldots Full time

    Job Description : As a Senior Data Engineer at ParallelDots, you will be responsible for overseeing the design and development of advanced data solutions. You will work closely with data scientists, analysts, and other stakeholders to ensure that our data infrastructure meets the highest standards of scalability, reliability, and : - Lead the design,...

  • Data Engineer

    4 weeks ago

    Anywhere in India/Multiple Locations, IN Paralleldots Full time

    Job Description :As a Senior Data Engineer at ParallelDots, you will be responsible for overseeing the design and development of advanced data solutions. You will work closely with data scientists, analysts, and other stakeholders to ensure that our data infrastructure meets the highest standards of scalability, reliability, and :- Lead the design,...

  • Data Engineer

    2 weeks ago

    Any Location, India Muoro Full time

    Responsibilities :- Design, develop, and maintain robust data pipelines using Apache Spark, Python, and other relevant technologies.- Implement data ingestion processes from various sources, ensuring data quality and integrity.- Optimize and tune data pipelines for performance and scalability.- Collaborate with data scientists and analysts to deploy machine...

  • Data Engineer

    4 weeks ago

    Any Location, IN Muoro Full time

    Responsibilities :- Design, develop, and maintain robust data pipelines using Apache Spark, Python, and other relevant technologies.- Implement data ingestion processes from various sources, ensuring data quality and integrity.- Optimize and tune data pipelines for performance and scalability.- Collaborate with data scientists and analysts to deploy machine...

  • Lead Data Engineer

    1 week ago

    Anywhere in India/Multiple Locations SeaCross Recruitment Services Full time

    Job Description : - Ability to support the development of infrastructural components of the LDH stack- Strong coding experience and knowledge of cloud infrastructure, expertise in Python, Spark, SQL (Postgres)- Strong expertise in Azure, Databricks, Gen2 Storage, Terraform- Strong experience working with restful APIs and Event-Driven Architecture - Pub/Sub...

  • Lead Data Engineer

    1 week ago

    Anywhere in India/Multiple Locations, IN SeaCross Recruitment Services Full time

    Job Description :- Ability to support the development of infrastructural components of the LDH stack- Strong coding experience and knowledge of cloud infrastructure, expertise in Python, Spark, SQL (Postgres)- Strong expertise in Azure, Databricks, Gen2 Storage, Terraform- Strong experience working with restful APIs and Event-Driven Architecture - Pub/Sub or...

  • Senior Data Engineer- Spark/Python

    1 month ago

    Anywhere in India,Multiple Locations,Bangalore,Chennai iMind Your Business Solutions Private Limited Full time

    We are hiring senior data engineers for the leaders in analytics and cloud applications, headquartered in San Francisco and with offices in India (Bangalore and Chennai).Our client provides strategic guidance and technology systems for clients wishing to solve their most complex and interesting business challenges involving cloud applications, big data,...

  • Senior Data Engineer- Spark/Python

    4 weeks ago

    Anywhere in India/Multiple Locations/Bangalore/Chennai, IN iMind Your Business Solutions Private Limited Full time

    We are hiring senior data engineers for the leaders in analytics and cloud applications, headquartered in San Francisco and with offices in India (Bangalore and Chennai).Our client provides strategic guidance and technology systems for clients wishing to solve their most complex and interesting business challenges involving cloud applications, big data,...

  • Xebia - Azure Data Engineer - Python/Spark/PySpark

    1 month ago

    Anywhere in India,Multiple Locations Xebia IT Architects India Pvt Ltd Full time

    Role - Azure Data Engineer - 4+ years of experience in IT Industry - 2+ years of experience with Azure Data Engineering Stack (Event Hub, Data Factory, Cosmos DB, Synapse, SQL DB, Databricks)- 2+ years of experience with Python/Spark/Pyspark- Excellent knowledge of SQL and coding skills- Good understanding of other Azure services like Azure Data Lake...

  • Senior Python Developer

    4 weeks ago

    Anywhere in India/Multiple Locations, IN Varite Full time

    Salary : 12-20 LPA Responsibilities :Develop pipeline objects using PySpark, Flink, Apache Spark, and Scala. Design and implement real-time data streams utilizing Flink and PySpark. Utilize AWS services such as Glue, EMR, and VPC for infrastructure management and deployment. Design and develop data pipeline architectures using Confluent Kafka, Spark, and...

  • Senior Python Developer

    2 weeks ago

    Anywhere in India/Multiple Locations Varite Full time

    Salary : 12-20 LPA Responsibilities : Develop pipeline objects using PySpark, Flink, Apache Spark, and Scala. Design and implement real-time data streams utilizing Flink and PySpark. Utilize AWS services such as Glue, EMR, and VPC for infrastructure management and deployment. Design and develop data pipeline architectures using Confluent Kafka, Spark,...

  • Data Engineering Lead

    2 weeks ago

    Anywhere in India/Multiple Locations VAST Consulting Full time

    Job Description : - Should have 8+ years of working experience.- Bachelor's or Master's degree in Computer Science, Engineering, or related field.- Demonstrated leadership experience leading cross-functional teams in the design and implementation of complex data solutions.- Experience in a data engineering role with strong understanding of graph...

  • Big Data Developer

    4 weeks ago

    Anywhere in India/Multiple Locations, IN Risk Resources LLP Full time

    Responsibilities :- Lead and mentor a team of Big Data engineers in designing, developing, and deploying big data solutions on GCP- Define the technical vision and roadmap for big data architecture within the organization- Architect and implement scalable, high-performance big data pipelines using technologies like MapReduce, Hive, Spark, and BigQuery-...

  • Big Data Developer

    1 month ago

    Anywhere in India,Multiple Locations Risk Resources LLP Full time

    Responsibilities :- Lead and mentor a team of Big Data engineers in designing, developing, and deploying big data solutions on GCP- Define the technical vision and roadmap for big data architecture within the organization- Architect and implement scalable, high-performance big data pipelines using technologies like MapReduce, Hive, Spark, and BigQuery-...

  • Senior Spark Developer

    1 month ago

    Bangalore,Anywhere in India,Multiple Locations CYBERSAPIENT TECHNOLOGIES PRIVATE LIMITED Full time

    About the job :Job Description / Duties & Responsibilities :- Develop, program, and maintain applications using the Apache Spark open-source framework.- Work with different aspects of the Spark ecosystem, including Spark SQL, DataFrames, Datasets, and streaming.- Spark Developer must have strong programming skills in Java, Scala, or Python.- Familiar with...

  • Lead Data Engineer

    2 weeks ago

    Bangalore/Anywhere in India/Multiple Locations Boyen Haddin Full time

    Job Description: - 8 - 10+ years' experience in developing scalable Big Data applications or solutions on distributed platforms.- Able to partner with others in solving complex problems by taking a broad perspective to identify.- innovative solutions.- Strong skills building positive relationships across Product and Engineering.- Able to influence and...

  • Application Developer

    2 weeks ago

    Anywhere in India,Multiple Locations Optivalue tek consulting Full time

    Summary : As an Application Developer, you will be responsible for designing, building, and configuring applications to meet business process and application requirements using Python and Apache Spark. Your typical day will involve working with PySpark, performing data analysis and munging to extract insights and patterns from large data sets.Roles &...