Data Engineer

4 days ago

Bengaluru, India XANDER CONSULTING AND ADVISORY PRIVATE LIMITED Full time

Job Title : Data Engineer - Azure Databricks, SQL, PySpark (5-12 years experience)

Location : Bangalore & Chennai, India

Job Description :

We are seeking an experienced Data Engineer with expertise in Azure Databricks, SQL, and PySpark to join our dynamic team in Bangalore or Chennai. As a Data Engineer, you will play a pivotal role in designing and implementing scalable data solutions, enabling the organization to leverage large datasets for advanced analytics and business intelligence. If you are passionate about building robust data pipelines, working with cloud technologies, and optimizing data workflows, we want to hear from you

Key Responsibilities :

Data Pipeline Development : Design, develop, and optimize robust ETL/ELT pipelines for processing large datasets using Azure Databricks, SQL, and PySpark.

Azure Cloud Services : Work extensively with Azure Data Lake, Azure Databricks, Azure SQL Database, and Azure Synapse Analytics to build scalable data solutions on the cloud.

Data Integration & Transformation : Integrate diverse data sources, ensuring seamless data flow from various systems into the data warehouse or lake using advanced data transformation techniques.

Performance Optimization : Continuously monitor and optimize the performance of data pipelines and jobs written in PySpark and SQL for efficiency, speed, and cost reduction.

SQL Querying & Analysis : Write complex SQL queries to extract, transform, and load (ETL) data from different sources and prepare datasets for analytics and reporting.

Collaboration : Work closely with data scientists, analysts, and business teams to ensure alignment on data requirements and deliverables.

Automation : Automate data workflows, reducing manual intervention and increasing data processing efficiency and reliability.

Big Data Processing : Leverage the power of distributed computing frameworks like PySpark and Databricks to process and analyze large-scale datasets effectively.

Data Security & Governance : Ensure adherence to data security, compliance, and governance standards in all data processes and workflows.

Skills & Qualifications :

Experience : 5-12 years of experience in data engineering or related roles with hands-on expertise in Azure Databricks, SQL, and PySpark.

Azure Expertise : Strong experience with Azure cloud services, particularly Azure Databricks, Azure Data Lake, Azure SQL Database, Azure Synapse Analytics, and Azure Data Factory.

Programming & Frameworks : Proficiency in PySpark and other data processing frameworks (e.g., Hadoop, Spark) to manage large-scale data pipelines and transformations.

SQL Skills : Advanced skills in SQL for data querying, data manipulation, and optimization.

Big Data Technologies : Hands-on experience with big data technologies and distributed data processing tools such as Spark and Hadoop.

ETL Process : Strong knowledge of ETL processes and tools, including experience designing and implementing efficient ETL workflows.

Problem Solving : Ability to troubleshoot complex data issues and optimize data pipelines for performance, scalability, and reliability.

Collaboration : Excellent teamwork and communication skills to work effectively with cross-functional teams, including data scientists, analysts, and stakeholders.

Data Modeling & Architecture : Experience with data modeling, designing scalable data architectures, and building data warehouses or lakes.

Education :

Bachelor's or Master's degree in Computer Science, Engineering, Information Technology, or a related field.

(ref:hirist.tech)

  • Gcp Data Science

    2 days ago

    Bengaluru, Karnataka, India NTT DATA Full time

    **Statistical Modeling**: Strong understanding of statistical techniques and experience leveraging BigQuery ML and Vertex AI for model development. **Machine Learning Algorithms**: Experience with GCP's pre-built ML APIs (e.g., Vision AI, Natural Language API) and custom model building using Vertex AI. **Feature Engineering**: Ability to create meaningful...

  • Ai Data Architect

    4 days ago

    Bengaluru, Karnataka, India NTT DATA Full time

    **Req ID**: 303823 We are currently seeking a AI Data Architect to join our team in Bangalore, Karnātaka (IN-KA), India (IN). Job Duties: Experience: - 10+ years of experience in Solution Architecting of Data & Analytics, AI/ML & Gen AI Solutions as Presales Consultant/ Business Development Manager/ Solution Architect **Responsibilities**: - Solid...

  • Data engineer

    2 days ago

    Bengaluru, India Recro Full time

    We are hiring !!Role - Data EngineerLocation - BangaloreExperience Required - 4+SummaryWe are seeking a highly skilled and motivated Data Engineer Sr. Associate to join ourdynamic team. The ideal candidate will have extensive experience in data engineering,with a strong focus on Databricks, Python, and SQL. As a Data Engineer Sr. Associate,you will play a...

  • Data Engineer

    10 hours ago

    Bengaluru, India Recro Full time

    We are hiring !!Role - Data EngineerLocation - BangaloreExperience Required - 4+SummaryWe are seeking a highly skilled and motivated Data Engineer Sr. Associate to join ourdynamic team. The ideal candidate will have extensive experience in data engineering,with a strong focus on Databricks, Python, and SQL. As a Data Engineer Sr. Associate,you will play a...

  • Data Engineer

    3 hours ago

    Bengaluru, India Recro Full time

    Associate - Data EngineerLocation: Bengaluru, IndiaSummaryWe are seeking a highly skilled and motivated Data Engineer Sr. Associate to join our dynamic team. The ideal candidate will have extensive experience in data engineering, with a strong focus on Databricks, Python, and SQL. As a Data Engineer Sr. Associate, you will play a crucial role in designing,...

  • Data Engineer

    6 hours ago

    Bengaluru, India Wipro Full time

    Job Description: Context: 10+ years - Data engineers are required to assist with transitioning and migration of the data pipelines running on Teradata to the Jungle asset developed on the Openshift platform. Tech responsibilities: • Migration of data pipelines from Teradata to Starburst (mpp sql query engine, open source version is known as...

  • Data Engineer

    4 days ago

    Bengaluru, India Recro Full time

    We are hiring !! Role - Data Engineer Location - Bangalore Experience Required - 4+ SummaryWe are seeking a highly skilled and motivated Data Engineer Sr. Associate to join ourdynamic team. The ideal candidate will have extensive experience in data engineering,with a strong focus on Databricks, Python, and SQL. As a Data Engineer Sr. Associate,you will play...

  • Data Engineer

    4 days ago

    Bengaluru, India Recro Full time

    We are hiring !! Role - Data Engineer Location - Bangalore Experience Required - 4+ SummaryWe are seeking a highly skilled and motivated Data Engineer Sr. Associate to join ourdynamic team. The ideal candidate will have extensive experience in data engineering,with a strong focus on Databricks, Python, and SQL. As a Data Engineer Sr. Associate,you will play...

  • Data Engineer

    2 days ago

    Bengaluru, India Recro Full time

    We are hiring !! Role - Data Engineer Location - Bangalore Experience Required - 4+ Summary We are seeking a highly skilled and motivated Data Engineer Sr. Associate to join our dynamic team. The ideal candidate will have extensive experience in data engineering, with a strong focus on Databricks, Python, and SQL. As a Data Engineer Sr. Associate, you...

  • Data Engineer

    3 hours ago

    Bengaluru, India Wipro Full time

    Job Description:Context: 10+ years - Data engineers are required to assist with transitioning and migration of the data pipelines running on Teradata to the Jungle asset developed on the Openshift platform.Tech responsibilities:• Migration of data pipelines from Teradata to Starburst (mpp sql query engine, open source version is known as Trinio/Presto)•...

  • Data Engineer

    21 hours ago

    Bengaluru, India Recro Full time

    Associate - Data Engineer Location: Bengaluru, IndiaSummary We are seeking a highly skilled and motivated Data Engineer Sr. Associate to join our dynamic team. The ideal candidate will have extensive experience in data engineering, with a strong focus on Databricks, Python, and SQL. As a Data Engineer Sr. Associate, you will play a crucial role in designing,...

  • Data Engineer

    14 hours ago

    Bengaluru, India Recro Full time

    Associate - Data Engineer Location: Bengaluru, IndiaSummary We are seeking a highly skilled and motivated Data Engineer Sr. Associate to join our dynamic team. The ideal candidate will have extensive experience in data engineering, with a strong focus on Databricks, Python, and SQL. As a Data Engineer Sr. Associate, you will play a crucial role in designing,...

  • Data Engineer

    8 hours ago

    Bengaluru, India Recro Full time

    Associate - Data Engineer Location: Bengaluru, India Summary We are seeking a highly skilled and motivated Data Engineer Sr. Associate to join our dynamic team. The ideal candidate will have extensive experience in data engineering, with a strong focus on Databricks, Python, and SQL. As a Data Engineer Sr. Associate, you will play a crucial role in...

  • Data Engineer

    1 day ago

    Bengaluru, India iO Associates - UKEU Full time

    Job Title: Data Engineer Location: Bangalore Salary: Negotiable on the current package iO Associates is seeking for skilled Data Engineers to build an innovative platform integrating risk management, research, trading analytics, and AI. This role suits those with expertise in developing robust data pipelines and an analytical mindset for enabling...

  • Data Engineer

    5 days ago

    Bengaluru, India iO Associates - UKEU Full time

    Job Title: Data EngineerLocation: BangaloreSalary: Negotiable on the current packageiO Associates is seeking for skilled Data Engineers to build an innovative platform integrating risk management, research, trading analytics, and AI. This role suits those with expertise in developing robust data pipelines and an analytical mindset for enabling...

  • Data Engineer

    21 hours ago

    Bengaluru, India Wipro Full time

    Job Description:Context: 10+ years - Data engineers are required to assist with transitioning and migration of the data pipelines running on Teradata to the Jungle asset developed on the Openshift platform.Tech responsibilities:• Migration of data pipelines from Teradata to Starburst (mpp sql query engine, open source version is known as Trinio/Presto)•...

  • Data Engineer

    8 hours ago

    Bengaluru, India Wipro Full time

    Job Description: Context: 10+ years - Data engineers are required to assist with transitioning and migration of the data pipelines running on Teradata to the Jungle asset developed on the Openshift platform. Tech responsibilities: • Migration of data pipelines from Teradata to Starburst (mpp sql query engine, open source version is known as Trinio/Presto)...

  • Data Engineer

    11 hours ago

    Bengaluru, India Tata Consultancy Services Full time

    Greetings from TATA Consultancy ServicesJob Openings at TCSSkill :Sr. Azure Data Engineer with data bricksExp range :5+8 yearsRole : Permanent RoleJob location :bangaloreCurrent location : bangaloreInterview Date: 25th jan 25(saturdayInterview mode : WALKIN INTERVIEWPls find the Job Description below.Job Description for Sr. Data Engineer Role–Data Engineer...

  • Data Engineer

    5 days ago

    Bengaluru, India Wipro Full time

    Immediate hiring for AWS/Azure Data Engineer positionExp: 8 to 12 yrsLocation: BangalorePlease find the below JD:-AWS Data Engineer:-Data Pipeline design and orchestrationAWS (S3, EMR, Lambda, Glue, Redshift, Lakeformation, firehose, Neptune, Quicksight)HadoopSpark/PySparkScala/Java/PythonAzure Data Engineer:-Azure (DataLake, ADB, ADF, ADLS, Functions,...

  • Data Engineer

    3 days ago

    Bengaluru, India Tata Consultancy Services Full time

    Greetings from TATA Consultancy Services Job Openings at TCS Skill :Sr. Azure Data Engineer with data bricks Exp range :5+8 years Role : Permanent Role Job location :bangalore Current location : bangalore Interview Date: 25th jan 25(saturday Interview mode : WALKIN INTERVIEW Pls find the Job Description below. Job Description for Sr. Data Engineer Role ...