Udemy - Senior Staff Software Engineer - Data Warehousing
3 weeks ago
Job Description :
Are you excited about building a Data Mesh platform on a global scale? Do you want to build a self-service, highly scalable platform to power Udemy's groundbreaking AI, ML, and data products? Udemy's Data org is looking for a self-driven, creative Data Mesh platform engineer passionate about changing the world by democratizing online education.
We are building highly available, scalable data infrastructure, services, tools, and libraries to serve critical data discovery, ML/data pipeline development & deployment, data security & compliance, and self-healing data operation needs.
Come join us and play a key role in developing and driving the adoption of our Data Mesh, data-centric micro-services, data engineering, and ML platform.
Responsibilities :
- Design, build, enhance, and support self-service data infrastructure as a platform, which would be used for Udemy's data mesh.
- Build robust, scalable, cloud-native, cost-optimized real-time and batch data integration pipelines in AWS with Airflow, Kafka, Databricks, Delta Lake, AWS services like S3, EMR, Athena, RDS, Redshift, EKS, etc., and Snowflake.
- Contribute towards Udemy's comprehensive data quality initiatives like data contracts, data validation, observability, and alerting
- Support data platform privacy with RBAC and encryption
- On-call rotation for business hours of data platforms with hundreds of data platform users
- 8+ years of experience in building distributed systems with Java, Scala, or Python
- 5+ years of experience managing big data platforms with storage (AWS S3), compute (AWS EMR or Databricks with Spark, AWS Athena or Presto), data warehouses (Redshift or Snowflake), streaming (Kafka or AWS Kinesis)
- 5+ years of experience in agile development methodologies & test-driven development (TDD)
- 5+ years of experience building and maintaining resilient real-time & batch data pipelines.
- Experience in building or using any one of the data quality tools (in-house built, SODA SQL, Anomalo, Monte Carlo, Great Expectations, or any other)
- Expertise in cloud data privacy tools in the area of authentication (AWS IAM/OKTA), encryption (AWS KMS/Safenet), authorization (Apache Ranger/Privacera)
- Strong grasp of object-oriented design patterns, algorithms, and data structures
- Good understanding of Data Mesh and Data Contract paradigm
- Experience with container systems (e.g., Docker) and container orchestration (e.g., ECS/EKS, Kubernetes)
- Active contributions to open-source projects are a plus
- Experience in ML Ops is a plus
- Experience in Terraform and Ansible
-
Senior Data Engineer
3 weeks ago
Chennai, Tamil Nadu, India iMind Your Business Solutions Private Limited Full timeJob Description : We are hiring Senior Data Engineer for the market leaders In Analytics & Cloud Applications, headquartered in San Francisco and have offices in India (Chennai, Bangalore, Hyderabad and Pune).Our client provides strategic guidance and technology systems for clients wishing to solve their most complex and interesting business challenges...
-
Senior Staff Software Engineer
3 weeks ago
Bangalore/Chennai, Tamil Nadu, India Cortex Consulting Pvt. Ltd. Full timeAbout the Role:We are seeking a motivated and experienced Senior Staff Software Engineer to join our dynamic team. In this critical role, you will play a key part in architecting, designing, and implementing security and transport-related technologies for our network security solutions. You will leverage your in-depth knowledge of networking protocols and...
-
Senior DevOps and SysOps Consultant/Engineer
2 days ago
Chennai, Tamil Nadu, India DATA SOFTWARE RESEARCH COMPANY Full timeApply for Senior DevOps And SysOps Consultant/Engineer DSRC, Career Progress Consultants in Chennai for 5 - 6 Year of Experience on
-
Senior Data Engineer
3 weeks ago
Chennai, Tamil Nadu, India hubwork Full timeSummary :We are looking for a data engineer who has experience in building data pipelines for large data platforms. This role will be working with a team building a large data platform catering to the needs of a wealth manager for their application and analytics needs. This role will be augmented with a customer of Intellect and other third parties in...
-
Data Engineer
3 weeks ago
Chennai, Tamil Nadu, India CliqHR Full timeJob Description :- 5 years of experience as a Data Engineer, with a strong proficiency in PySpark, Python, SQL, Databricks, and AWS.- Hands-on experience designing and implementing data pipelines for large-scale data processing and analytics.- Strong understanding of data modeling, data warehousing concepts, and database technologies.- Proficiency in working...
-
Senior Manager
1 month ago
Chennai, Tamil Nadu, India Tata Motors Finance Full timeJob Requirements Job Title: Senior Manager - SCM Warehousing at Tata Motors Finance, ChennaiCompany Name: Tata Motors FinanceLocation: ChennaiSalary: Best in IndustryEmployment Type: Full Time, PermanentJob Description:Are you an experienced Supply Chain Management professional looking to take the next step in your career? We have a fantastic opportunity for...
-
Data Engineer, VP
3 weeks ago
Chennai, Tamil Nadu, India NatWest Digital X Full timeJoin us as a Data EngineerYou'll be the voice of our customers, using data to tell their stories and put them at the heart of all decision-makingWe'll look to you to drive the build of effortless, digital first customer experiencesIf you're ready for a new challenge and want to make a far-reaching impact through your work, this could be the opportunity...
-
Data Engineer, AVP
1 week ago
Chennai, Tamil Nadu, India NatWest Digital X Full timeJoin us as a Data EngineerWe're looking for someone to build effortless, digital first customer experiences to help simplify our organisation and keep our data safe and secureDay-to-day, you'll develop innovative, data-driven solutions through data pipelines, modelling and ETL design while inspiring to be commercially successful through insightsIf you're...
-
Senior Data Engineer
3 weeks ago
Chennai, Tamil Nadu, India Softcrylic Full timeWho We Are :For 20 years, we have been working with organizations large and small to help solve business challenges through technology. We bring a unique combination of engineering and strategy to Make Data Work for organizations. Our clients range from the travel and leisure industry to publishing, retail, and banking. The common thread between our clients...
-
Data Architect
3 weeks ago
Chennai, Tamil Nadu, India CliqHR Full timeJob Description :A minimum of 10+ years of experience in data architecture and analytics. - Databricks : Proficiency in Databricks for data processing, machine learning, and analytics. - Cloud Platforms : Strong experience with any one or more cloud platforms such as AWS, Azure, GCP, and the ability to integrate Databricks with them. - Data Engineering :...
-
Aerial Telecom
3 weeks ago
Chennai, Tamil Nadu, India Aerial Telecom Solutions Full timeJob Title : Data EngineerLocation : Chennai (Work from Office)Experience : 5 to 8 yearsJob Description : We are seeking an experienced Data Engineer to join our team in Chennai. The ideal candidate will have 5 to 8 years of experience in data engineering, with a strong background in building data pipelines, data warehousing, and data management...
-
Bangalore/Chennai, Tamil Nadu, India CGI Information Systems and Management Consultants Full timeLooking for experienced AWS Data Engineer / Developer / Lead proficient in Lambda or Kinesis or Redshift, adept at architecting and deploying scalable serverless applications. Specialized in real-time data processing, data warehousing, and API development, with a strong focus on security, optimization, and collaborative problem-solving.Job Title : AWS Data...
-
Senior Software Engineer
3 weeks ago
Chennai, Tamil Nadu, India Fullfily Full timeDescription :We are seeking a highly experienced and motivated Senior Software Engineer to join our dynamic team. As a Senior Software Engineer, you will play a vital role in the development and maintenance of our software products. You will collaborate with cross-functional teams, including product managers, designers, and other engineers, to deliver...
-
Software Engineer
2 weeks ago
Chennai, Tamil Nadu, India NatWest Group Full timeJoin us as a Software EngineerThis is an opportunity for a driven Software Engineer to take on an exciting new career challengeDay-to-day, you'll build a wide network of stakeholders of varying levels of seniorityIt's a chance to hone your existing technical skills and advance your careerWe're offering this role at associate levelWhat you'll do In your new...
-
data engineer
10 hours ago
Chennai, Tamil Nadu, India Atos Full time[Data Modernization];[Azure Data Platform] [Microsoft Power Platform];[Power BI - Platform] Expirience of 7-8 years of experience working on MS Power BI Projects Must experience with all the components of MS Power BI including Excellent understanding of Continuousflow graphs and its implementation. Must Experience on creating Project plans with appropriate...
-
Senior MDM Data Engineer
3 weeks ago
Chennai, Tamil Nadu, India Resource Tree Global Services Full timeJob Name : Senior MDM Data EngineerYears of Experience : 3Job Description : As a Senior MDM Data Engineer with hands-on experience in Informatica Cloud MDM, you will be responsible for leading and executing end-to-end master data management (MDM) initiatives within our organization. Leveraging your deep technical expertise and practical experience, you will...
-
Software Engineer
3 weeks ago
Chennai, Tamil Nadu, India NatWest Digital X Full timeJoin us as a Software EngineerThis is an opportunity for a driven Software Engineer to take on an exciting new career challengeDay-to-day, you'll build a wide network of stakeholders of varying levels of seniorityIt's a chance to hone your existing technical skills and advance your careerWe're offering this role at associate levelWhat you'll do In your new...
-
Senior Developer/Tech Lead
3 weeks ago
Chennai/Hyderabad, Tamil Nadu, India Huquo Full timeRole Description: Sr. Developer - K2View Fabric- 3B Location: : The ideal candidate is a hands-on K2View Developer with experience in developing ETL scripts and Data Warehousing for real-world use cases. They must be at ease working with on-prem Big Data ecosystem in an agile environment with little supervision. The person should be a self-motivated person...
-
Senior Azure Data Engineer
3 weeks ago
Chennai, Tamil Nadu, India Resourcetree Full timeJob Name : Senior Data Engineer - AzureYears of Experience : 4- 7 yrsJob Description : We are looking for a skilled and experienced Senior Azure Developer to join our team! As part of the team, you will be involved in the implementation of the ongoing and new initiatives for our company.If you love learning, thinking strategically, innovating, and helping...
-
Software Engineer
1 week ago
Chennai, Tamil Nadu, India NatWest Digital X Full timeJoin us as a Software EngineerThis is an opportunity for a driven Software Engineer to take on an exciting new career challengeDay-to-day, you'll build a wide network of stakeholders of varying levels of seniorityIt's a chance to hone your existing technical skills and advance your careerWhat you'll do In your new role, you'll engineer and maintain...