Data Engineering Expert

1 day ago

guntur, India beBeeData Full time

Data Engineer Role OverviewAbout the JobAs a Data Engineer, you will design, build, and optimize data pipelines and real-time systems that power AI-driven decisioning and analytics.Main ResponsibilitiesData Pipeline Development: Develop and maintain scalable ETL/ELT pipelines using Python, Airflow, and dbt to ensure seamless data flow.Real-Time Streaming: Build and optimize real-time streaming pipelines using Kafka, RabbitMQ, Spark, and event-driven microservices to facilitate instant data processing.AI-assisted Coding: Leverage AI coding agents (Claude Code, Cursor AI, GitHub Copilot) to accelerate data pipeline development, data transformation, and optimization for improved efficiency.Data Storage Solutions: Design and manage data storage solutions with PostgreSQL, ClickHouse, and Elasticsearch to support data scalability and performance.Data Modeling: Define and implement data modeling best practices for efficient and scalable data storage to reduce data complexity.Data Infrastructure: Ensure high availability, performance, and security of data infrastructure on cloud platforms to guarantee business continuity.BI-ready Datasets: Design and implement BI-ready datasets in ClickHouse optimized for OLAP workloads and reporting use cases to enhance business insights.Data Transformation Pipelines: Develop and manage denormalized schemas and data models to support analytics and amCharts-based reporting for data visualization.Query Performance Optimization: Optimize query performance, partitioning, and indexing strategies in ClickHouse to handle large-scale analytical workloads efficiently.Data Quality and Observability: Implement best practices for data quality, validation, and observability to ensure accurate data analysis.Data Ingestion: Work on data ingestion using Airbyte or other connector frameworks to integrate diverse data sources.Martin Integration: Implement data transformation pipelines supporting analytical and ML workloads for enhanced data value.Cloud and DevOps Collaboration: Collaborate with cloud and DevOps teams for containerized deployments (Docker, Kubernetes, AWS) to streamline deployment processes.ML Model APIs: Support integration of ML workflows and model APIs into production pipelines for seamless data-driven decision-making.Requirements6+ years of hands-on experience in data engineering and distributed systems to deliver complex projects.

  • Data engineer

    6 days ago

    Guntur, India Straive Full time

    Job Title: Data Engineer Location: Remote Job Type: Full time Years of Experience- 6-9 years About Straive: Straive is a market leading Content and Data Technology company providing data services, subject matter expertise, & technology solutions to multiple domains. Data Analytics & Al Solutions, Data Al Powered Operations and Education & Learning form the...

  • Data Engineer

    1 hour ago

    Guntur, India BayOne Solutions Full time

    The Opportunity:We are seeking a highly experienced Data Engineer to join our MarTech team and play a pivotal role in driving innovation within our microservices architecture, with a strong emphasis on data engineering and back-end systems. You will be responsible for leading the development, optimization, and maintenance of both data pipelines and back-end...

  • Azure data engineer

    1 week ago

    Guntur, India Techolution Full time

    We are seeking a skilled Azure Data Engineer with hands-on experience in modern data engineering tools and platforms within the Azure ecosystem . The ideal candidate will have a strong foundation in data integration, transformation, and migration , along with a passion for working on complex data migration projects . Job Title: Azure Data Engineer Location:...

  • Azure data engineer

    1 week ago

    Guntur, India Techolution Full time

    We are seeking a skilled Azure Data Engineer with hands-on experience in modern data engineering tools and platforms within the Azure ecosystem . The ideal candidate will have a strong foundation in data integration, transformation, and migration , along with a passion for working on complex data migration projects . Job Title: Azure Data Engineer Location:...

  • Data Engineer

    2 days ago

    guntur, India Digitalzone Full time

    About the RoleAs a Data Engineer, you will design, build, and optimize data pipelines and real-time systems that power AI-driven decisioning and analytics.ResponsibilitiesDevelop and maintain scalable ETL/ELT pipelines using Python, Airflow, and dbtBuild and optimize real-time streaming pipelines using Kafka, RabbitMQ, Spark and event-driven...

  • Lead data engineer

    4 days ago

    Guntur, India NexGen Tech Solutions Full time

    Job Title: Lead Data Engineer – Wealth Management Location: (Noida/Gurgaon/Remote) Experience: 10+ years Must Have: Wealth Management Domain, Fabric, Data Lake, Python Job Summary: We are seeking a highly skilled Lead Data Engineer (Azure) with deep expertise in the Wealth Management domain to design, implement, and manage scalable data solutions....

  • Lead data engineer

    3 days ago

    Guntur, India NexGen Tech Solutions Full time

    Job Title: Lead Data Engineer – Wealth Management Location: (Noida/Gurgaon/Remote) Experience: 10+ years Must Have: Wealth Management Domain, Fabric, Data Lake, Python Job Summary: We are seeking a highly skilled Lead Data Engineer (Azure) with deep expertise in the Wealth Management domain to design, implement, and manage scalable data solutions....

  • Azure Senior Data Engineer

    7 days ago

    Guntur, India Whatjobs IN C2 Full time

    Greetings from TCS!!!! TCS is hiring for AWS Data Engineer Experience : 5 to 10yrs Location: Pan India Mode of Driving : Virtual Date: 15-Nov-25 (Saturday) Job Description: Technical Experience 4+ years’ experience in Data Platform Engineering Extensive coding experience including Python/Pyspark Experience developing in Azure with key data technologies...

  • Chief Data Scientist

    6 days ago

    Guntur, India TalentBridge Full time

    Job Title: Associate Director, Data ScientistLocation: RemoteKey Responsibilities:Data Analysis and Preprocessing: Analyze and preprocess diverse datasets relevant to the mortgage industry, ensuring data quality and relevance for model training.Model Development and Fine-Tuning: Research and implement state-of-the-art NLP models, focusing on pre-training as...

  • ▷ Urgent: Senior Data Engineer

    1 week ago

    Guntur, India Guidanz Inc Full time

    About BI Connector BI Connector is the industry leading solution for integrating Oracle Fusion Cloud data into modern BI platforms like Power BI, Tableau, and Data Warehouse, without complex ETL. Our Data Architecture and Reporting Solutions team extends this mission by helping enterprises unlock the full value of their Oracle Fusion ERP, SCM, HCM data...