Udemy - Senior Staff Software Engineer - Data Warehousing

3 weeks ago


Chennai, Tamil Nadu, India Udemy Full time

Job Description :


Are you excited about building a Data Mesh platform on a global scale? Do you want to build a self-service, highly scalable platform to power Udemy's groundbreaking AI, ML, and data products? Udemy's Data org is looking for a self-driven, creative Data Mesh platform engineer passionate about changing the world by democratizing online education.

We are building highly available, scalable data infrastructure, services, tools, and libraries to serve critical data discovery, ML/data pipeline development & deployment, data security & compliance, and self-healing data operation needs.

Come join us and play a key role in developing and driving the adoption of our Data Mesh, data-centric micro-services, data engineering, and ML platform.

Responsibilities :

  • Design, build, enhance, and support self-service data infrastructure as a platform, which would be used for Udemy's data mesh.
  • Build robust, scalable, cloud-native, cost-optimized real-time and batch data integration pipelines in AWS with Airflow, Kafka, Databricks, Delta Lake, AWS services like S3, EMR, Athena, RDS, Redshift, EKS, etc., and Snowflake.
  • Contribute towards Udemy's comprehensive data quality initiatives like data contracts, data validation, observability, and alerting
  • Support data platform privacy with RBAC and encryption
  • On-call rotation for business hours of data platforms with hundreds of data platform users
Must have :
  • 8+ years of experience in building distributed systems with Java, Scala, or Python
  • 5+ years of experience managing big data platforms with storage (AWS S3), compute (AWS EMR or Databricks with Spark, AWS Athena or Presto), data warehouses (Redshift or Snowflake), streaming (Kafka or AWS Kinesis)
  • 5+ years of experience in agile development methodologies & test-driven development (TDD)
  • 5+ years of experience building and maintaining resilient real-time & batch data pipelines.
  • Experience in building or using any one of the data quality tools (in-house built, SODA SQL, Anomalo, Monte Carlo, Great Expectations, or any other)
  • Expertise in cloud data privacy tools in the area of authentication (AWS IAM/OKTA), encryption (AWS KMS/Safenet), authorization (Apache Ranger/Privacera)
  • Strong grasp of object-oriented design patterns, algorithms, and data structures
Bonus points if you have :
  • Good understanding of Data Mesh and Data Contract paradigm
  • Experience with container systems (e.g., Docker) and container orchestration (e.g., ECS/EKS, Kubernetes)
  • Active contributions to open-source projects are a plus
  • Experience in ML Ops is a plus
  • Experience in Terraform and Ansible

(ref:hirist.tech)
  • Senior Data Engineer

    3 weeks ago


    Chennai, Tamil Nadu, India iMind Your Business Solutions Private Limited Full time

    Job Description : We are hiring Senior Data Engineer for the market leaders In Analytics & Cloud Applications, headquartered in San Francisco and have offices in India (Chennai, Bangalore, Hyderabad and Pune).Our client provides strategic guidance and technology systems for clients wishing to solve their most complex and interesting business challenges...


  • Bangalore/Chennai, Tamil Nadu, India Cortex Consulting Pvt. Ltd. Full time

    About the Role:We are seeking a motivated and experienced Senior Staff Software Engineer to join our dynamic team. In this critical role, you will play a key part in architecting, designing, and implementing security and transport-related technologies for our network security solutions. You will leverage your in-depth knowledge of networking protocols and...


  • Chennai, Tamil Nadu, India DATA SOFTWARE RESEARCH COMPANY Full time

    Apply for Senior DevOps And SysOps Consultant/Engineer DSRC, Career Progress Consultants in Chennai for 5 - 6 Year of Experience on

  • Senior Data Engineer

    3 weeks ago


    Chennai, Tamil Nadu, India hubwork Full time

    Summary :We are looking for a data engineer who has experience in building data pipelines for large data platforms. This role will be working with a team building a large data platform catering to the needs of a wealth manager for their application and analytics needs. This role will be augmented with a customer of Intellect and other third parties in...

  • Data Engineer

    3 weeks ago


    Chennai, Tamil Nadu, India CliqHR Full time

    Job Description :- 5 years of experience as a Data Engineer, with a strong proficiency in PySpark, Python, SQL, Databricks, and AWS.- Hands-on experience designing and implementing data pipelines for large-scale data processing and analytics.- Strong understanding of data modeling, data warehousing concepts, and database technologies.- Proficiency in working...

  • Senior Manager

    1 month ago


    Chennai, Tamil Nadu, India Tata Motors Finance Full time

    Job Requirements Job Title: Senior Manager - SCM Warehousing at Tata Motors Finance, ChennaiCompany Name: Tata Motors FinanceLocation: ChennaiSalary: Best in IndustryEmployment Type: Full Time, PermanentJob Description:Are you an experienced Supply Chain Management professional looking to take the next step in your career? We have a fantastic opportunity for...

  • Data Engineer, VP

    3 weeks ago


    Chennai, Tamil Nadu, India NatWest Digital X Full time

    Join us as a Data EngineerYou'll be the voice of our customers, using data to tell their stories and put them at the heart of all decision-makingWe'll look to you to drive the build of effortless, digital first customer experiencesIf you're ready for a new challenge and want to make a far-reaching impact through your work, this could be the opportunity...

  • Data Engineer, AVP

    1 week ago


    Chennai, Tamil Nadu, India NatWest Digital X Full time

    Join us as a Data EngineerWe're looking for someone to build effortless, digital first customer experiences to help simplify our organisation and keep our data safe and secureDay-to-day, you'll develop innovative, data-driven solutions through data pipelines, modelling and ETL design while inspiring to be commercially successful through insightsIf you're...

  • Senior Data Engineer

    3 weeks ago


    Chennai, Tamil Nadu, India Softcrylic Full time

    Who We Are :For 20 years, we have been working with organizations large and small to help solve business challenges through technology. We bring a unique combination of engineering and strategy to Make Data Work for organizations. Our clients range from the travel and leisure industry to publishing, retail, and banking. The common thread between our clients...

  • Data Architect

    3 weeks ago


    Chennai, Tamil Nadu, India CliqHR Full time

    Job Description :A minimum of 10+ years of experience in data architecture and analytics. - Databricks : Proficiency in Databricks for data processing, machine learning, and analytics. - Cloud Platforms : Strong experience with any one or more cloud platforms such as AWS, Azure, GCP, and the ability to integrate Databricks with them. - Data Engineering :...

  • Aerial Telecom

    3 weeks ago


    Chennai, Tamil Nadu, India Aerial Telecom Solutions Full time

    Job Title : Data EngineerLocation : Chennai (Work from Office)Experience : 5 to 8 yearsJob Description : We are seeking an experienced Data Engineer to join our team in Chennai. The ideal candidate will have 5 to 8 years of experience in data engineering, with a strong background in building data pipelines, data warehousing, and data management...


  • Bangalore/Chennai, Tamil Nadu, India CGI Information Systems and Management Consultants Full time

    Looking for experienced AWS Data Engineer / Developer / Lead proficient in Lambda or Kinesis or Redshift, adept at architecting and deploying scalable serverless applications. Specialized in real-time data processing, data warehousing, and API development, with a strong focus on security, optimization, and collaborative problem-solving.Job Title : AWS Data...


  • Chennai, Tamil Nadu, India Fullfily Full time

    Description :We are seeking a highly experienced and motivated Senior Software Engineer to join our dynamic team. As a Senior Software Engineer, you will play a vital role in the development and maintenance of our software products. You will collaborate with cross-functional teams, including product managers, designers, and other engineers, to deliver...

  • Software Engineer

    2 weeks ago


    Chennai, Tamil Nadu, India NatWest Group Full time

    Join us as a Software EngineerThis is an opportunity for a driven Software Engineer to take on an exciting new career challengeDay-to-day, you'll build a wide network of stakeholders of varying levels of seniorityIt's a chance to hone your existing technical skills and advance your careerWe're offering this role at associate levelWhat you'll do In your new...

  • data engineer

    10 hours ago


    Chennai, Tamil Nadu, India Atos Full time

    [Data Modernization];[Azure Data Platform] [Microsoft Power Platform];[Power BI - Platform] Expirience of 7-8 years of experience working on MS Power BI Projects Must experience with all the components of MS Power BI including Excellent understanding of Continuousflow graphs and its implementation. Must Experience on creating Project plans with appropriate...


  • Chennai, Tamil Nadu, India Resource Tree Global Services Full time

    Job Name : Senior MDM Data EngineerYears of Experience : 3Job Description : As a Senior MDM Data Engineer with hands-on experience in Informatica Cloud MDM, you will be responsible for leading and executing end-to-end master data management (MDM) initiatives within our organization. Leveraging your deep technical expertise and practical experience, you will...

  • Software Engineer

    3 weeks ago


    Chennai, Tamil Nadu, India NatWest Digital X Full time

    Join us as a Software EngineerThis is an opportunity for a driven Software Engineer to take on an exciting new career challengeDay-to-day, you'll build a wide network of stakeholders of varying levels of seniorityIt's a chance to hone your existing technical skills and advance your careerWe're offering this role at associate levelWhat you'll do In your new...


  • Chennai/Hyderabad, Tamil Nadu, India Huquo Full time

    Role Description: Sr. Developer - K2View Fabric- 3B Location: : The ideal candidate is a hands-on K2View Developer with experience in developing ETL scripts and Data Warehousing for real-world use cases. They must be at ease working with on-prem Big Data ecosystem in an agile environment with little supervision. The person should be a self-motivated person...


  • Chennai, Tamil Nadu, India Resourcetree Full time

    Job Name : Senior Data Engineer - AzureYears of Experience : 4- 7 yrsJob Description : We are looking for a skilled and experienced Senior Azure Developer to join our team! As part of the team, you will be involved in the implementation of the ongoing and new initiatives for our company.If you love learning, thinking strategically, innovating, and helping...

  • Software Engineer

    1 week ago


    Chennai, Tamil Nadu, India NatWest Digital X Full time

    Join us as a Software EngineerThis is an opportunity for a driven Software Engineer to take on an exciting new career challengeDay-to-day, you'll build a wide network of stakeholders of varying levels of seniorityIt's a chance to hone your existing technical skills and advance your careerWhat you'll do In your new role, you'll engineer and maintain...