▷ (Only 24h Left) Data Engineer/Senior Data Engineer

1 day ago

Gurgaon, India Cognitio Analytics Inc Full time
Experience:

1 to 8 years - DE/ Sr. DE/ AM /Manager

Must Have:

- SQL- Python/ Pyspark- Azure Databricks, ADLS Gen 2 Blob Storage, Key Vaults, BitBucket, Version control, Azure DevOps- ETL, Building Data Pipelines, Data Warehousing, Data Modelling and Governance, Data Security- Agile Practices, SDLC- Data Exploration, Data Quality metrics, Historical data Management, Good problem-solving skill- Excellent Comm Skills- Exp in Data Engineering domain: 2+ Years.- Multi years’ experience with Azure-Databricks ecosystem and Py spark.- Ability to write clean, concise and organized Py spark code.- Ability to break down the project into executable steps, prepare a DFD and execute the same- Propose innovative DE solutions to achieve business objectives.- Quick on his feet, good at tech and logically complex communication- Good Knowledge of ADF, Docker /containerization.

Good to have:

- Event Hubs- Any kind of health care background eg, - Pharma- Azure Data Factory, Cosmos DB, Power BI, Azure Synapse- Competitive coding and knows most Py spark syntax by heart.

Job Description:

We are seeking a highly skilled Azure Cloud Data Engineer to join our dynamic team. As an Azure Cloud Data Engineer, you will be responsible for designing, implementing, and managing cloud-based data solutions using Microsoft Azure. The ideal candidate should have a strong background in data engineering, with hands-on experience in Azure services and tools.

Essential Job Duties & Responsibilities:

- Set up workflows and orchestration processes to streamline data pipelines and ensure efficient data movement within the Azure ecosystem.- Create and configure compute resources within Databricks, including All-Purpose and SQL Compute and Job Clusters to support data processing and analysis.- Set up and manage Azure Data Lake (ADLS) Gen 2 storage accounts and establish a seamless integration with Databricks Workspace for data ingestion and processing.- Create and manage Service Principals, key vaults to securely authenticate and authorize access to Azure resources.- Utilize ETL (Extract, Transform, Load) techniques to design and implement data warehousing solutions and ensure compliance with data governance policies.- Develop highly automated ETL scripts for data processing.- Scale infrastructure resources based on workload requirements, optimizing performance and cost-efficiency.- Profile new data sources in a different format including CSVs, JSONs etc.- Apply problem-solving skills to address complex business and technical challenges, such as data quality issues, performance bottlenecks, and system failures.- Demonstrate excellent soft skills and the ability to effectively communicate and collaborate with clients, stakeholders, and cross-functional teams.- Implement Continuous Integration/Continuous Deployment (CI/CD) practices to automate the deployment and testing of data pipelines and infrastructure changes.- Delivering tangible value very rapidly, collaborating with diverse teams of varying backgrounds and disciplines.- Codifying best practices for future reuse in the form of accessible, reusable patterns, templates, and code bases.- Manage timely appropriate communication and relationship with clients, partners and other stakeholders.- Create and manage periodic reporting of project execution status and other trackers in standard accepted formats.

Required Competencies:

- Strong proficiency in SQL for data manipulation, querying, and optimization.- Proficiency in Python and PySpark for data engineering and data processing tasks.- Solid understanding of the Microsoft Azure tooling for large-scale data engineering efforts and deployments is highly preferred.- Hands-on experience with Azure Databricks, including data ingestion, transformation, and analysis.- Ability to set up and manage Azure Data Lake Storage (ADLS) Gen 2 accounts, and familiarity with data lake architecture and best practices.- Knowledge of Azure Key Vaults for securely storing and managing cryptographic keys, secrets, and certificates.- An ‘engineering’ mindset, willing to make rapid, pragmatic decisions to improve performance, accelerate progress or magnify impact.- Comfort with working with distributed teams on code-based deliverables, using version control systems and code reviews.- Use of agile and DevOps practices for project and software management including continuous integration and continuous delivery.- Ability to conduct data analysis, investigation, and lineage studies to document and enhance data quality and access.- Proven track record of scaling infrastructure to meet performance and scalability requirements.- Strong analytical and problem-solving skills, with the ability to troubleshoot complex technical issues.- Direct experience having built and deployed robust, complex production systems.- Proficiency in Event Hubs and Azure Data Factory, particularly in the context of data streaming and transfer.

Qualifications:

- Bachelor’s degree in computer science, Engineering, or a quantitative discipline.- 2+ years of professional experience as an Azure Cloud Data Engineer or a similar role.- Excellent communication skills, both verbal and written.- Strong attention to detail and the ability to work in a fast-paced environment.

  • ▷ [Only 24h Left] Senior Data Engineer

    2 days ago

    Gurgaon, India True Tech Professionals Full time

    Senior Data EngineerLocation: Gurgaon (Hybrid Mode)Experience: 5 years and above.Immediate Joiners Preferred (Notice Period: 7 to 10 days)We are seeking a skilled Senior Data Engineer to join our dynamic team. The ideal candidate should have a strong background in data engineering with expertise in Python, Pyspark, AWS Glue, Athena, Redshift, and SQL. The...

  • ▷ [Only 24h Left] Associate Data Engineer

    1 week ago

    Gurgaon, India Majid Al Futtaim Full time

    Job Summary:We are seeking a dedicated Data Engineering and BI Support Associate with expertise in Data Warehouse / Big Data, Azure platform, SAP Data Services, SAP Business Objects to join our technical support team. In this role, you will be responsible for supporting the day-to-day operations of data pipelines, troubleshooting technical issues, and...

  • Big Data Engineer

    1 month ago

    Gurgaon, Haryana, India ACS Consultants Full time

    **Job Title:** Big Data Engineer**Job Summary:We are seeking an experienced Big Data Engineer to join our team at ACS Consultants. As a key member of our data science team, you will be responsible for designing, developing, and managing large-scale data processing systems using Hadoop, Hive, and Spark.Main Responsibilities:• Design and implement scalable...

  • Data Engineer

    2 weeks ago

    Gurgaon, India NAB Full time

    PFB JD for Data Engineer -Essential capabilitiesAdvanced or demonstrable experience in SQL, Python,Experience with Source code control (GitHub) and JenkinsAbility to design, develop, execute, complete ETLAbility to articulate complex technical issues and desired outcomes of system enhancements- Proven analytical skills and evidence-based decision making-...

  • Senior Data Engineer

    1 month ago

    Chennai/Gurgaon/Gurugram/Delhi NCR/Hyderabad, India Eucloid Data Solutions Pvt Ltd Full time

    About Eucloid Data SolutionsEucloid Data Solutions is a fast-growing AI-consulting firm that enables business growth for its clients through advanced Data Analytics, Machine Learning and Data engineering. We offer bespoke, full stack and data-driven approach to solve highly complex and large-scale business problems for some of the biggest Fortune 500...

  • Senior Data Engineer

    1 month ago

    Chennai/Gurgaon/Gurugram/Delhi NCR/Hyderabad, India Eucloid Data Solutions Pvt Ltd Full time

    About the Role:Eucloid Data Solutions Pvt Ltd is seeking a highly skilled Senior Data Engineer to join our Data Platform team. The successful candidate will have hands-on expertise in Databricks and will be responsible for designing and building data infrastructure to support various business applications.Key Responsibilities:Design, implementation, and...

  • Senior Data Engineer

    4 weeks ago

    Chennai/Gurgaon/Gurugram/Delhi NCR/Hyderabad, India Eucloid Data Solutions Pvt Ltd Full time

    **Job Title:** Senior Data Engineer - Databricks Expert**About Eucloid Data Solutions Pvt Ltd**Eucloid Data Solutions Pvt Ltd is a fast-growing AI-consulting firm that enables business growth for its clients through advanced Data Analytics, Machine Learning, and Data engineering. Our team supports various business applications by developing data...

  • Senior Data Engineer

    1 month ago

    Gurgaon, Haryana, India FedEx Full time

    Data Engineer - SeniorFedEx is seeking a highly skilled Senior Data Engineer to join our team. As a Senior Data Engineer, you will be responsible for designing, developing, and maintaining large-scale data pipelines and architectures that support our business operations.Key Responsibilities:Develop and maintain scalable data pipelines and...

  • Data Engineer Lead for Azure Services

    7 days ago

    Gurgaon, Haryana, India MINDTEL GLOBAL PRIVATE LIMITED Full time

    Job DescriptionWe are seeking an experienced Data Engineer Lead to join our team at MINDTEL GLOBAL PRIVATE LIMITED. This role is responsible for designing, developing, and maintaining robust data pipelines using Azure Data Factory (ADF) to ingest, transform, and load data into data warehouses and data lakes.Key ResponsibilitiesDesign and develop efficient...

  • ▷ (Only 24h Left) Senior Data Scientist

    2 weeks ago

    Gurgaon, India Zupee Full time

    About ZupeeZupee is India’s fastest growing Technology backed Behavioral Science company. We are innovating Skill-Based Gaming with a mission to become the most trusted and responsible entertainment company in the world. We have been constantly focusing on innovation of indigenous games to entertain the mass.Our strategy is to invest in our people & user...

  • Eucloid - Senior Data Engineer - Azure Databricks

    6 months ago

    Chennai/Gurgaon/Gurugram, India Eucloid Data Solutions Pvt Ltd Full time

    Job Role : Sr. Data Engineer (Databricks) About Eucloid :Eucloid Data Solutions is a fast-growing AI-consulting firm that enables business growth for its clients through advanced Data Analytics, Machine Learning and Data engineering. Eucloid offers bespoke, full stackand datadriven approach to solve highly complex and large-scale business problems for some...

  • Senior Data Engineer

    1 month ago

    Gurgaon, Haryana, India Habilelabs Pvt. Ltd. Full time

    Job Title: Senior Data EngineerAbout Habilelabs Pvt. Ltd.Habilelabs Pvt. Ltd. is a leading technology company that specializes in developing innovative solutions for the data engineering space. We are seeking a highly skilled and motivated Senior Data Engineer to join our development team in Gurgaon.Job Description:We are looking for a talented Senior Data...

  • Senior Software Engineer

    4 weeks ago

    Gurgaon, Haryana, India NTT DATA Full time

    Job Title: Senior Software EngineerAs a Senior Software Engineer at {company}, you will be responsible for designing, developing, and maintaining complex software applications. You will work closely with cross-functional teams to ensure timely delivery of high-quality software products.Responsibilities:Design and develop software applications using a variety...

  • Senior Data Engineer

    1 month ago

    Gurgaon, Haryana, India Elite recruitments Full time

    Job Title: Senior Data EngineerJob Summary:Elite recruitments is hiring a seasoned data engineer to join our team. As a senior data engineer, you will be responsible for building efficient data pipelines using tools like Spark, Cloud native tools like EMR, Azure data factory, Informatica, Abinitio. You will also be responsible for understanding the need for...

  • Senior Data Engineer

    3 days ago

    Gurgaon, India True Tech Professionals Full time

    Senior Data EngineerLocation: Gurgaon (Hybrid Mode)Experience: 5 years and above.Immediate Joiners Preferred (Notice Period: 7 to 10 days)We are seeking a skilled Senior Data Engineer to join our dynamic team. The ideal candidate should have a strong background in data engineering with expertise in Python, Pyspark, AWS Glue, Athena, Redshift, and SQL. The...

  • Senior Data Engineer

    1 month ago

    Gurgaon, Haryana, India Petonic Infotech Pvt. Ltd. Full time

    Job TitleSenior Data Engineer Position Overview:Petonic Infotech Pvt. Ltd. is seeking a skilled Senior Data Engineer to play a pivotal role in our AI-driven project development. The ideal candidate will design, develop, and maintain robust data pipelines, manage APIs, and collaborate with data teams to ensure optimal data quality and access. This key role...

  • Senior Data Engineer

    4 weeks ago

    Gurgaon/Gurugram/Chennai/Hyderabad/Delhi NCR, India Eucloid Data Solutions Pvt Ltd Full time

    About Eucloid Data SolutionsEucloid Data Solutions is a pioneering AI-consulting firm that empowers business growth through cutting-edge Data Analytics, Machine Learning, and Data engineering. Our bespoke, full-stack approach helps solve complex and large-scale business problems for top Fortune 500 organizations. With a presence across multiple continents...

  • ▷ (Only 24h Left) Automation Test Analyst ( Data Engineering)

    2 weeks ago

    Gurgaon, India Coforge Full time

    Role: Automation Tester ( Data Engineering)Skills: ETL Testing, Unit test, Pytest, Python , Java , seleniumExperience: 5+ yearsLocation: GurugramNotice Period: Immediate to 30 would be preferredWe are seeking a skilled Automation Tester to join our dynamic team. The ideal candidate will have a strong background in data engineering, proficiency in Python, and...

  • Eucloid - Data Engineer - Azure Databricks

    6 months ago

    Gurgaon/Gurugram/Chennai, India Eucloid Data Solutions Pvt Ltd Full time

    Job Description - Data Engineer (Databricks)About Eucloid : Eucloid Data Solutions is a fast-growing AI-consulting firm that enables business growth for its clients through advanced Data Analytics, Machine Learning and Data engineering. Eucloid offers bespoke, full stackand datadriven approach to solve highly complex and large-scale business problems for...

  • Data Engineering Manager

    1 month ago

    Gurgaon, Haryana, India United Airlines Full time

    About the RoleUnited Airlines is seeking a highly skilled Senior Manager to lead our Data Engineering team. As a key member of our Digital Technology team, you will be responsible for designing, developing, and maintaining our enterprise-wide data and analytics environment.Key ResponsibilitiesLead a team of engineers in the development of big data...