Transformative Data Engineer

5 hours ago

Lucknow, Uttar Pradesh, India beBeeDataEngineering Full time US$ 1,00,000 - US$ 1,40,000
As a data engineering professional, you will play a key role in designing and developing scalable data pipelines and enterprise data solutions. The job requires strong technical skills and expertise in various tools and technologies.A successful candidate should have experience working with Azure Data Factory (ADF) and Databricks for batch and real-time data processing. They should also be able to integrate data from various structured and unstructured sources including SQL Server, Azure SQL Database, Azure Data Lake Storage (ADLS), and external APIs.The ideal candidate should possess excellent problem-solving skills and the ability to work collaboratively with data analysts, data scientists, and business stakeholders to define and deliver data solutions.This position involves building and maintaining data models, data marts, and data warehouses using SQL Server and Azure Synapse Analytics. Additionally, the candidate should be able to write efficient and optimized SQL queries, stored procedures, views, and triggers in SQL Server.They should also have experience using Databricks (Spark with Python/Scala) to process large datasets for transformation and analytics.Data quality, integrity, security, and compliance are crucial aspects of this role, and the candidate should be able to ensure these aspects are maintained across the pipeline using data validation, monitoring, and auditing techniques.A Bachelor's degree in Computer Science, Engineering, Information Systems, or related field is required, along with 5+ years of experience in data engineering or related roles. Strong understanding of data warehousing concepts and excellent problem-solving skills are also essential.Responsibilities:
  1. Design and develop ETL/ELT pipelines using Azure Data Factory (ADF) and Databricks for batch and real-time data processing.
  2. Integrate data from various structured and unstructured sources including SQL Server, Azure SQL Database, Azure Data Lake Storage (ADLS), and external APIs.
  3. Build and maintain data models, data marts, and data warehouses using SQL Server and Azure Synapse Analytics.
  4. Write efficient and optimized SQL queries, stored procedures, views, and triggers in SQL Server.
  5. Use Databricks (Spark with Python/Scala) to process large datasets for transformation and analytics.
  6. Ensure data quality, integrity, security, and compliance across the pipeline using data validation, monitoring, and auditing techniques.
  7. Collaborate with data analysts, data scientists, and business stakeholders to define and deliver data solutions.
  8. Implement CI/CD pipelines using Azure DevOps.
  9. Monitor and optimize the performance of data pipelines and queries across Azure and SQL Server environments.
Requirements:Azure Data Services, Databricks, Microsoft SQL Server, strong understanding of data warehousing concepts, and excellent problem-solving skills are essential for this role. We offer a competitive salary, opportunities for growth and development, and a collaborative work environment. If you are passionate about data engineering and want to make a meaningful impact, we encourage you to apply.

  • Transformative Data Governance Expert

    3 days ago

    Lucknow, Uttar Pradesh, India beBeeDataGovernance Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Unlock the Power of Data GovernanceAs a Data Engineer _ MS Fabric, you will play a pivotal role in transforming data insights into business value by designing and implementing effective governance structures for Power BI environments. You will analyze usage patterns, workspace structure, and licensing to identify areas of improvement.Main...

  • Data Engineer

    2 weeks ago

    Lucknow, Uttar Pradesh, India LTIMindtree Full time

    Title : Data Engineer (Azure Synapse ) Experience: 5 to 8 years Loc: Pan India (Pune/Mumbai/Hyderabad/banagalore/Kolkata/Chennai/Noida) Interview Mode: 1 Virtual , 1 F2F Primary Skill – Experience with Azure Synapse (Synapse pipeline, IR, notebook). Secondary Skill – Databricks, PySpark, ADF Job Description : Design build and maintain scalable...

  • Azure Data Solutions Engineer

    1 day ago

    Lucknow, Uttar Pradesh, India beBeeDataEngineer Full time ₹ 10,00,000 - ₹ 16,00,000

    Job Title: Data Engineer - Azure Databricks DeveloperWe are seeking a highly skilled Data Engineer with expertise in Azure Databricks and PySpark to join our team. The ideal candidate will have hands-on experience with Databricks, Delta Lake, and Python scripting.Key Responsibilities:Design, develop, and deploy data pipelines using Databricks and Azure...

  • Transformation Data Analyst

    8 hours ago

    Lucknow, Uttar Pradesh, India beBeeBusiness Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Job Description:As a seasoned Business Analyst, you will be responsible for driving business transformation initiatives by leveraging technology to address a wide range of business needs.The ideal candidate will possess excellent skills in static/reference data management, including experience with reputable data vendors such as Bloomberg and Refinitiv.With...

  • Senior Data Engineer

    4 days ago

    Lucknow, Uttar Pradesh, India Infogain Full time

    Job Summary:Big Data Engineer (Lead): As a Big Data Engineer (Lead), you will be responsible for leading a team of big data engineers. You will work closely with clients and team members to understand their requirements and develop architectures that meet their needs. You will also be responsible for providing technical leadership and guidance to your...

  • Data Architect

    2 days ago

    Lucknow, Uttar Pradesh, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Job SummaryWe are seeking a highly skilled Data Engineer to join our team.The ideal candidate will have 5+ years of experience in ETL development and data engineering, with strong expertise in Snowflake and data pipeline tools (FiveTran, DBT Cloud, HighTouch).They will also be proficient in SQL, Python, and AWS data services, with experience working with...

  • Data Engineer

    4 weeks ago

    Lucknow, Uttar Pradesh, India Bizdeed HR Solutions Pvt Full time

    Role : Data EngineerJob Location : Lucknow (On-Site)Experience : Min.4+ yrs as Data engineer with Python, SQL and pipeline experienceIMMEDIATE JOINERS ARE PREFERREDRole Responsibilities :- Analyze and organize raw data - Build data systems and pipelines- Prepare data for prescriptive and predictive modeling- Combine raw information from different sources-...

  • Generative AI Engineer

    3 weeks ago

    Lucknow, Uttar Pradesh, India Data Design Full time

    The selected candidate will be working for our Indian JV - D2AI Labs at our customer's premises in Riyadh, Saudi Arabia.ResponsibilitiesKey Responsibilities:•⁠ ⁠Lead Agent Development: Architect, build, and deploy the central interface agent that connects and coordinates specialized AI agents.•⁠ ⁠Agent Orchestration: Implement reliable...

  • Lead Big Data Engineer

    4 days ago

    Lucknow, Uttar Pradesh, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 25,00,000

    Job Opportunity:In this role, you will support the Data Engineering team in setting up the Data Lake on Cloud and implementing standardized Data Models.You will develop data pipelines for new sources, data transformations within the Data Lake, implementing GraphQL, working on NoSQL databases, CI/CD, and data delivery as per business requirements.Key...

  • Chief Data Platform Architect

    4 days ago

    Lucknow, Uttar Pradesh, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    We're seeking an experienced senior data engineer to design and build large-scale data platforms on Google Cloud.Key Responsibilities:Design robust data pipelines for ingestion, transformation, and analyticsMandatory strong experience in GCP ecosystemWork with Big Data tools: Hadoop, Spark, KafkaHands-on experience in Databricks (Unity Catalog, Workflow,...