Sr. Azure Data Engineer

4 weeks ago

Pune, India Fractal Full time

It's fun to work in a company where people truly BELIEVE in what they are doing

JOB DESCRIPTION:

If you are an extraordinary developer and who loves to push the boundaries to solve complex business problems using creative solutions, then we wish to talk with you. As a Data Engineer - Azure, you will work in the Technology team that helps deliver our Data Engineering offerings at large scale to our Fortune clients worldwide. The role is responsible for innovating, building, and maintaining technology services.


RESPONSIBILITIES:
• Be an integral part of large-scale client business development and delivery engagements.
• Develop the software and systems needed for end-to-end execution on large projects.
• Work across all phases of SDLC and use Software Engineering principles to build scaled solutions.
• Build the knowledge base required to deliver increasingly complex technology projects.
• Developing Data Pipelines (Batch/Streaming)
• Developing Complex data transformations
• ETL Orchestration
• Data Migration
• Develop and Maintain Datawarehouse / Data Lakes

EXPERIENCE:
• 6 to 9 years of experience in setting up data pipelines to injest streaming and batch data from many sources
• Experience on optimized data collection procedures
• Good experience in performing extract, transform and load (ETL) processes to make the data more usable with cloud infrastructures
• Expertise in data storage structures, data mining, and data cleansing
• Hands-on experience using Big Data components (Hadoop, Spark) & programming languages.
• Experience in using, or migrating to cloud databases
• Proficient in relational database design and development
• Proficient in Data warehousing concepts and up to date Data Integration patterns/technologies
• Experience in Azure data lake, Azure SQL Database, Data Bricks, Azure SQL Data warehouse and Azure Data factory.
• Experience in Developing Spark applications using Spark - SQL in Databricks for data extraction, transformation, and aggregation from multiple file formats for analyzing & transforming the data to uncover insights.
• Systematic problem-solving approach with strong communication skills and a sense of ownership and drive

TECHNICAL SKILLS:
• Strong Experience in Azure Cloud
• Experience with distributed data/computing tools: Map/Reduce, Hadoop, Hive, Spark
• Strong knowledge in Azure Analytics Products like Data Factory, HD Insights, Azure Data Lake Storage, Azure SQL Datawarehouse & Data Bricks
• Strong in Python Programming Skills
• Strong experience in networking and security measures
• Good knowledge in using NoSQL databases like MongoDB, PostgreSQL and Cassandra running huge volumes of data
• Proficient in with SQl Databases like MSSql, Postgres.
• Expertise in Data Warehousing – Hadoop, MapReduce, HIVE, PIG, Apache Spark, Kafka
• Good understanding of Spark Architecture including Spark Core, Spark SQL, Data Frames, Spark Streaming, Driver Node, Worker Node, Stages, Executors and Tasks.
• Good understanding of Big Data Hadoop and Yarn architecture along with various Hadoop Demons such as Job Tracker, Task Tracker, Name Node, Data Node, Resource/Cluster Manager, and Kafka (distributed stream-processing).

JOB RESPONSIBILITES:
• Building and maintaining the data architecture for data science projects.
• Extract Transform and Load data from sources Systems to Azure Data Storage services using a combination of Azure Data Factory, T-SQL, Spark SQL and U-SQL Azure Data Lake Analytics.
• Data Ingestion to one or more Azure Services - (Azure Data Lake, Azure Storage, Azure SQL, Azure DW) and processing the data in In Azure Data bricks.
• Define table schema and quickly adapt with the pipeline.
• Improve data foundational procedures, integrating new data management technologies and softwares into the existing system, building data collection pipelines
• Identify valuable data sources and automate collection processes.
• Implement concepts of programming such as data structures, error handing, data manipulation and I/O processing. Implements database concepts and practices including definition and query language.
• Implement adequate networking and security measures for the data pipeline.
• Implement monitoring solution for the data pipeline.
• Develop Spark applications using Pyspark and Spark-SQL for data extraction, transformation and aggregation from multiple file formats for analyzing & transforming the data to uncover insights into the customer usage patterns.
• Support the design, and implement data engineering solutions
• Maintain excellent documentation for understanding and accessing data storage
• Work independently as well as in teams to deliver transformative solutions to clients
• Be proactive and constantly pay attention to the scalability, performance and availability of our systems
• Establishes privacy/security hierarchy and regulates access
• Undertake preprocessing of structured and unstructured data
• Support in collecting, organizing, and interpreting data along with fellow colleagues.
• Collaborate with engineering and product development teams

EDUCATION:
• B.E/B.Tech/M.Tech in Computer Science or related technical degree OR Equivalent

If you like wild growth and working with happy, enthusiastic over-achievers, you'll enjoy your career with us

Not the right fit? Let us know you're interested in a future opportunity by clickingin the top-right corner of the page or create an account to set up email alerts as new job postings become available that meet your interest

  • Senior Azure Data Engineer

    5 days ago

    Pune, Maharashtra, India Yash Technologies Full time

    Sr.Software Engineer - Azure Experience: - 7 years of Experience What you should have? Azure Data Factory, Azure Dev Ops CI/CD, Azure Data Lake, Azure Synapse, Azure Databricks, Py Spark, Git Hub.Need 4+ Years of experience Good analytical skills with excellent knowledge of SQL.Well versed with Azure Services Must have experience and knowledge on ADF, ADLS,...

  • Senior Azure Data Engineer

    3 weeks ago

    pune, India YASH Technologies Full time

    Sr. Software Engineer - AzureExperience: 4 - 7 years of ExperienceWhat you should have?Azure Data Factory, Azure DevOps CI/CD, Azure Data Lake, Azure Synapse, Azure Databricks, PySpark, GitHub.Need 4+ Years of experience Good analytical skills with excellent knowledge of SQL.Well versed with Azure ServicesMust have experience and knowledge on ADF, ADLS, Blob...

  • Sr. Azure Data Engineer

    4 weeks ago

    Pune, India Fractal Full time

    It's fun to work in a company where people truly BELIEVE in what they are doing!JOB DESCRIPTION:If you are an extraordinary developer and who loves to push the boundaries to solve complex business problems using creative solutions, then we wish to talk with you. As a Data Engineer - Azure, you will work in the Technology team that helps deliver our Data...

  • Senior Azure Data Engineer

    2 months ago

    Pune, India YASH Technologies Full time

    Sr. Software Engineer - Azure Experience: 4 - 7 years of Experience What you should have? Azure Data Factory, Azure DevOps CI/CD, Azure Data Lake, Azure Synapse, Azure Databricks, PySpark, GitHub. Need 4+ Years of experience Good analytical skills with excellent knowledge of SQL. Well versed with Azure Services Must have experience and knowledge on ADF,...

  • Senior Azure Data Engineer

    2 months ago

    Pune, India YASH Technologies Full time

    Sr. Software Engineer - Azure Experience: 4 - 7 years of Experience What you should have? Azure Data Factory, Azure DevOps CI/CD, Azure Data Lake, Azure Synapse, Azure Databricks, PySpark, GitHub. Need 4+ Years of experience Good analytical skills with excellent knowledge of SQL. Well versed with Azure Services Must have experience and knowledge on ADF,...

  • Senior Azure Data Engineer

    3 weeks ago

    Pune, India YASH Technologies Full time

    Sr. Software Engineer - Azure Experience: 4 - 7 years of Experience What you should have? Azure Data Factory, Azure DevOps CI/CD, Azure Data Lake, Azure Synapse, Azure Databricks, PySpark, GitHub. Need 4+ Years of experience Good analytical skills with excellent knowledge of SQL. Well versed with Azure Services Must have experience and knowledge on ADF,...

  • Senior Azure Data Engineer

    3 weeks ago

    Pune, India YASH Technologies Full time

    Sr. Software Engineer - Azure Experience: 4 - 7 years of Experience What you should have? Azure Data Factory, Azure DevOps CI/CD, Azure Data Lake, Azure Synapse, Azure Databricks, PySpark, GitHub. Need 4+ Years of experience Good analytical skills with excellent knowledge of SQL. Well versed with Azure Services Must have experience and knowledge on...

  • Senior Azure Data Engineer

    3 weeks ago

    Pune, India YASH Technologies Full time

    Sr. Software Engineer - Azure Experience: 4 - 7 years of Experience What you should have? Azure Data Factory, Azure DevOps CI/CD, Azure Data Lake, Azure Synapse, Azure Databricks, PySpark, GitHub. Need 4+ Years of experience Good analytical skills with excellent knowledge of SQL. Well versed with Azure Services Must have experience and knowledge on ADF,...

  • Sr. Data Engineer

    2 months ago

    Pune, India MICHELIN Full time

    Sr. Data Engineer- - - - - - - - - - - -A Data Engineer is a data professionals with extensive knowledge and expertise in “big data” technologies and frameworks that works with Python/Scala/Java. They are software engineers who design, build, integrate data from various resources, and manage big data. Experienced in writing spark code with python or...

  • Sr Azure Data Engineer

    5 days ago

    Pune, Maharashtra, India Mpowering HR Solutions Full time

    We are looking for an Azure Data Engineer with 5+ years of experience to join the team for one of UKs most wellknown client. You will use various methods to transform raw data into useful data systems. Overall, you'll strive for efficiency by aligning data systems with business goals. To succeed in this data engineering position, you should have strong...

  • Sr. Data Engineer

    5 days ago

    Pune, Maharashtra, India MICHELIN Full time

    Sr. Data Engineer A Data Engineer is a data professionals with extensive knowledge and expertise in "big data" technologies and frameworks that works with Python/Scala/Java. They are software engineers who design, build, integrate data from various resources, and manage big data. Experienced in writing spark code with python or Scala, Sql queries, Linux/Unix...

  • Sr. Data Engineer

    2 months ago

    Pune, India MICHELIN Full time

    Sr. Data Engineer - - - - - - - - - - - - A Data Engineer is a data professionals with extensive knowledge and expertise in “big data” technologies and frameworks that works with Python/Scala/Java. They are software engineers who design, build, integrate data from various resources, and manage big data. Experienced in writing spark code with python or...

  • Sr. Data Engineer

    2 months ago

    pune, India MICHELIN Full time

    Sr. Data Engineer - - - - - - - - - - - - A Data Engineer is a data professionals with extensive knowledge and expertise in “big data” technologies and frameworks that works with Python/Scala/Java. They are software engineers who design, build, integrate data from various resources, and manage big data. Experienced in writing spark code with python or...

  • Sr. Data Engineer

    5 days ago

    Pune, Maharashtra, India MICHELIN Full time

    Sr. Data Engineer A Data Engineer is a data professionals with extensive knowledge and expertise in "big data" technologies and frameworks that works with Python/Scala/Java. They are software engineers who design, build, integrate data from various resources, and manage big data. Experienced in writing spark code with python or Scala, Sql queries,...

  • Sr. Data Engineer Azure Databricks

    2 weeks ago

    Pune, India Fusemachines Full time

    About Fusemachines Fusemachines is a leading AI strategy, talent, and education services and products provider. Founded by Sameer Maskey Ph.D., Adjunct Associate Professor at Columbia University, Fusemachines has a core mission of democratizing AI. With a presence in 4 countries (Nepal, United States, Canada, and Dominican Republic and more than 400...

  • Sr. Data Engineer Azure Databricks

    4 weeks ago

    Pune, India Fusemachines Full time

    About Fusemachines Fusemachines is a leading AI strategy, talent, and education services and products provider. Founded by Sameer Maskey Ph.D., Adjunct Associate Professor at Columbia University, Fusemachines has a core mission of democratizing AI. With a presence in 4 countries (Nepal, United States, Canada, and Dominican Republic and more than 400...

  • Sr. Data Engineer Azure Databricks

    5 days ago

    Pune, Maharashtra, India Fusemachines Full time

    About Fusemachines Fusemachines is a leading AI strategy, talent, and education services and products provider. Founded by Sameer Maskey Ph.D., Adjunct Associate Professor at Columbia University, Fusemachines has a core mission of democratizing AI. With a presence in 4 countries (Nepal, United States, Canada, and Dominican Republic and more than 400...

  • Azure Data Engineer

    2 weeks ago

    pune, India EXL Full time

    Position - Senior Data EngineerWork Location – Pune/ Hyderabad / Gurgaon / HybridExperience - 5+ YearsMust have Skills:Minimum 4Y+ on Azure cloud services.Proficiency in using Azure Synapse, Azure Storage, and Azure Data Factory for building data solutions.Strong understanding of metadata-driven approaches and data governance principles.Excellent...

  • Azure Data Engineer

    3 weeks ago

    Pune, India EXL Full time

    Work Location– All EXL Locations (Hybrid)Experience- 5+ YearsMust have Skills:Minimum 4Y+ on Azure cloud services.Proficiency in using Azure Synapse, Azure Storage, and Azure Data Factory for building data solutions.Strong understanding of metadata-driven approaches and data governance principles.Excellent problem-solving skills and ability to work in a...

  • Azure Data Engineer

    3 weeks ago

    Pune, India EXL Full time

    Work Location– All EXL Locations (Hybrid)Experience- 5+ YearsMust have Skills:Minimum 4Y+ on Azure cloud services.Proficiency in using Azure Synapse, Azure Storage, and Azure Data Factory for building data solutions.Strong understanding of metadata-driven approaches and data governance principles.Excellent problem-solving skills and ability to work in a...