Data Engineer

3 weeks ago

Any Location, IN Blueridge Global Full time

Are you passionate about building state-of-the-art data platforms and powering the next generation of compute and AI applications, we'd love to hear from you. This is an exciting opportunity to leverage your expertise in distributed computing frameworks to make a significant impact as we push the boundaries of supply chain planning and eventually adoption of Gen AI.

JOB BRIEF :

We are seeking an experienced Data Engineer to join our team and lead the development of a cutting-edge data platform. The platform will leverage distributed computing frameworks such as Apache Spark, Databricks, and Snowflake to enable near real time supply chain planning, eventually leading to advanced analytics, insights into data with the adoption of Generative AI (GenAI) technologies across our product base.

KEY RESPONSIBILITIES :

As a Lead Data Engineer, the candidate would be responsible for :

- Design and build a highly scalable, fault-tolerant data platform optimized for distributed computing and large-scale data processing

- Implement data pipelines and ETL/ELT processes using distributed computing frameworks to efficiently ingest, transform, and load massive datasets from various sources

- Leverage cloud data platforms to enable seamless data sharing, near-zero maintenance, and fast analytics on structured and semi-structured data

- Collaborate with data scientists, machine learning engineers, and software developers to understand data requirements and build solutions to power GenAI applications

- Optimize distributed computing jobs and queries for maximum performance and cost efficiency

- Implement data governance, security, and compliance best practices

- Provide guidance on distributed computing architecture and mentor junior data engineers

QUALIFICATIONS :

- 5+ years of experience as a Data Engineer working with big data technologies.

- Strong proficiency in SQL, Python programming and data modeling techniques.

- Deep expertise in distributed computing principles and frameworks (e.g., Apache Spark), including SQL, streaming, and optimizing jobs for scale and efficiency.

- Hands-on experience with developing and deploying distributed computing applications using cloud-based platforms (e.g., AWS EMR, Azure HDInsight, or equivalent).

- Strong understanding of cloud data platform architectures and best practices for ELT/ETL, data sharing, and query optimization (e.g., AWS Athena, AWS Glue, Azure Synapse Analytics, or equivalent).

- Experience enabling application engineers to build applications leveraging the data platform through APIs and abstractions.

- Experience with orchestration frameworks like Apache Airflow and data streaming technologies like Kafka

- Experience building and optimizing data pipelines for machine learning applications.

- Knowledge of data modelling, data warehousing, and schema design.

- Familiarity with public cloud platforms such as AWS, Azure, or GCP.

- Excellent problem-solving and communication skills.


- Bachelor's or Master's degree in Computer Science (Preferred), Engineering, or a related field

(ref:hirist.tech)

  • Senior Network Engineer

    3 weeks ago

    Any Location, IN Frontline Data Solutions Full time

    Who We Are :Frontline Data Solutions provides web-based (SaaS) environmental, health, and safety software to some of the world's leading chemical, oil & gas, and manufacturing companies. Frontline's work culture combines the fast-paced, entrepreneurial culture of a startup software company with the stability and professionalism that reflects our...

  • Data Engineer

    5 months ago

    Any Location, IN Infometry Full time

    Job Description :We are seeking a skilled Data Engineer to join our dynamic team. As a Data Engineer, you will be responsible for designing, developing, and maintaining data pipelines and infrastructure to support our data warehousing and analytics initiatives. You will work closely with cross-functional teams to understand business requirements, design...

  • Data Engineer

    3 weeks ago

    Any Location, IN tlnt Full time

    Job Description :- Design, build, and maintain data pipelines in the Azure environment, ensuring smooth operation in both batch and real-time processing.- Optimize Azure data infrastructure to facilitate accurate data extraction, transformation, and loading from a variety of data sources.- Develop and automate ETL processes using tools like Pyspark, Azure...

  • Principal Data

    3 weeks ago

    Any Location, IN Employee Hub LLP Full time

    Position Overview : We are seeking an experienced and highly skilled Data and AI Principal Software Engineer to lead the design, development, and deployment of innovative data-driven and AI solutions. The ideal candidate will have a strong background in software engineering, data management, machine learning, and generative AI, with the ability to architect...

  • Senior Data Integration Engineer

    3 weeks ago

    Any Location, IN Employee Hub LLP Full time

    Job Description :- Design, implement, and manage data integration processes to consolidate data from various sources into our data warehouse. - Ensure seamless data flow and integration across systems.- Utilize Snowflake for data storage and processing, optimizing data architecture, and improving query performance. - Configure and manage Snowflake...

  • Lead Data Engineer

    3 weeks ago

    Any Location, IN Zorba Consulting India Pvt. Ltd. Full time

    Job Description :As a Lead Data Engineer, you will play a crucial role in driving data-driven initiatives within our organization. You will be responsible for designing, developing, and implementing scalable data solutions that support our business objectives. You will work closely with data scientists, analysts, and business stakeholders to ensure data...

  • Azure Data Engineer

    3 weeks ago

    Any Location, IN Salorix Consulting Full time

    Job Description :Key Responsibilities :- Design, build, and maintain scalable data pipelines using Azure Data Factory, Azure Databricks, and Azure Data Lake.- Develop and optimize data processing workflows using Pyspark and Python, ensuring high performance and reliability.- Implement streaming data solutions using Azure Event Hub or Apache Kafka to manage...

  • AI Automation Engineer

    3 weeks ago

    Any Location, IN Ermin Systems Private Limited Full time

    Role Description : This is a remote contract role for an AI Automation Engineer at Ermin Systems. The AI Automation Engineer will be responsible for troubleshooting, process control, electrical engineering, automation, and robotics tasks on a day-to-day basis. Qualifications : Experience :- 8+ years of experience in analytics engineering, with a strong...

  • Azure Data Engineer

    3 weeks ago

    Any Location, IN Catalyst IQ Full time

    Job Description :- Proven experience in designing, building, and maintaining ETL pipelines. - Strong proficiency in Python programming for data manipulation, transformation, and automation.Nice to Have :- Experience with Azure data services, including Azure Data Factory, Azure Databricks, and Azure SQL Database.- Knowledge of Terraform for infrastructure as...

  • Big Data Engineer

    3 weeks ago

    Any Location, IN First Phoenics Solutions Full time

    Job Description :- Develop and implement data systems and pipelines using programming languages such as Java, Scala, Python, C#, and SQL.- Design and build interfaces to support data integration and processing.- Utilize big data platforms and technologies for data ingestion, processing, ETL (Extract, Transform, Load), and telemetry ingestion.- Experience...

  • Techolution - Associate Google Cloud Platform Data Engineer

    3 weeks ago

    Any Location, IN Techolution Full time

    Job Description : We are looking for a Google Cloud Data Engineer who will help us build a highly scalable and reliable platform to match our exponential growth. As a Google Cloud Data Engineer, you will be responsible for building a solid back end infrastructure which will enable data delivery in near real-time using next-gen technologies.Title : Google...

  • AWS Data Engineer

    6 days ago

    Any Location, IN CAI Info India Pvt Ltd. Full time

    looking for only immediate joiners.We are looking for a Data Engineer to Design and Develop Data Lakes. This position will be full-time and remote.\What Youll Do :- Design and develop data lakes, manage data flows that integrate information from various sources into a common data lake platform through an ETL Tool- Code and manage delta lake implementations...

  • Senior Data Extraction Engineer

    3 weeks ago

    Any Location, IN Infnitrix Consulting Pvt Ltd Full time

    Job Description- Design, develop, and implement data extraction systems using Large LanguageModels (LLMs) such as GPT-4.- Collaborate with data scientists, engineers, and product managers to understanddata extraction requirements and objectives.- Fine-tune and customize LLMs for specific data extraction tasks, ensuring highaccuracy and efficiency.- Create...

  • Data Scientist

    3 weeks ago

    Any Location, IN Infnitrix Consulting Pvt Ltd Full time

    Key Responsibilities :- Analyze complex datasets to draw meaningful insights and inform decision-making.- Utilize Python programming for data analysis and model development.- Communicate key findings clearly and effectively to stakeholders and researchers.- Apply strong analytical skills to solve problems and address business needs.- Contribute to...

  • Data Scientist/Analyst

    3 months ago

    Any Location, IN Yo HR Consultancy Full time

    Role : Data Scientist/AnalystExperience : 2 - 15 YearsLocation : Anywhere in the World - Permanent RemoteContract Duration : 6-12 monthsWork timings which is primarily, overlap is required from 8 AM to 1 PM PST. The remaining 3 hours can be accommodated either before 8 AM PST or after 1 PM PSTOpportunity : Full Time, 8 hours, 6 monthsMust-Have :-...

  • Enterprise Data Architect

    3 weeks ago

    Any Location, IN Intraedge Technologies Ltd. Full time

    Job Description :- Design, develop, test, and deploy AI models and systems using various frameworks and tools.- Monitor and evaluate the performance and accuracy of AI models and systems in production.- Identify and troubleshoot issues related to data quality, model drift, scalability, and reliability.- Optimize and automate the AI model lifecycle using...

  • Data Scientist

    3 weeks ago

    Any Location, IN Calyptus Full time

    Job Description :Roles and Responsibilities : - Lead and mentor a team of data scientists and NLP specialists. - Manage multiple NLP projects from conception to deployment. - Collaborate with product managers, engineers, and stakeholders to understand requirements and deliver solutions. - Ensure timely delivery of projects while maintaining high-quality...

  • Business Intelligence Lead

    3 weeks ago

    Any Location, IN tlnt Full time

    Job Description : - Your responsibility will be to successfully execute the product delivery plan within your functional domain (Strategic Revenue Management (SRM) Route to Market (RTM).- You will support the Data Product Manager in the day-to-day implementation activities of the data product.- Coordination between various working groups, including business...

  • Aeonovatech - Senior Azure AI Engineer - Python Programming

    3 weeks ago

    Any Location, IN AEONOVATECH PRIVATE LIMITED Full time

    About the Role :We are seeking a highly skilled Senior Azure AI Engineer to join our team and contribute to the design, development, and deployment of cutting-edge AI solutions. You will be responsible for leveraging Azure AI services and OpenAI technologies to build intelligent applications.Responsibilities :- Design and develop AI models and algorithms...

  • Genpact - Senior ML Engineer/Principal Consultant - Spark/Python

    3 weeks ago

    Any Location, IN GENPACT India Private Limited Full time

    Job Description : Inviting applications for the role of Principal Consultant-Senior ML Engineer! The ML Engineer will focus on supporting the Growth Data Sciences team in data integration, model deployment and output integration with customer facing web and app applications. Responsibilities :- Developing functionality in Data Science Management Platform...