Senior Data Engineer for Large-Scale Data Platforms

5 days ago


Bengaluru, Karnataka, India Luxoft Full time
Job Summary

We are seeking a Senior Data Ops Engineer to develop a scalable data collection, storage, and distribution platform. The ideal candidate will have expertise in Snowflake and AWS, as well as experience with web scraping, containerization, and DevOps.

Key Responsibilities
  • Design and implement large-scale data platforms using Snowflake and AWS
  • Develop web scraping solutions using scripts, APIs, and tools
  • Maintain existing pipelines and enhance them for new requirements
  • Onboard new data providers and lead data migration projects
Requirements
  • Bachelor's degree in Computer Science or related field
  • 8+ years of experience in data engineering with focus on scalability and performance
  • Expertise in Snowflake and AWS, including IAM roles and security best practices
  • Experience with containerization (Docker, Kubernetes) and DevOps tools (Jenkins)
  • Strong understanding of Linux and SQL
  • Excellent communication skills and ability to work in a team environment
Nice-to-Have Skills
  • Market data projects or capital markets experience
  • Airflow knowledge and experience
Benefits
  • Competitive salary: $120,000 - $150,000 per year
  • Opportunity to work with cutting-edge technologies and collaborate with experienced professionals
  • Flexible work arrangements and remote work options


  • Bengaluru, Karnataka, India LinkedIn Full time

    At LinkedIn, we're passionate about empowering professionals to achieve their goals. As a Senior Engineering Leader, you'll play a critical role in shaping the future of our data systems. This is an exciting opportunity to join a dynamic team and drive innovation in a rapidly evolving industry.About the RoleWe're seeking an experienced engineering leader to...


  • Bengaluru, Karnataka, India Microsoft Full time

    Microsoft is a company where passionate innovators come to collaborate, envision what can be and take their careers further. This is a world of more possibilities, more innovation, more openness, and the sky is the limit thinking in a cloud-enabled world.Microsoft's Azure Data engineering team is leading the transformation of analytics in the world of data...


  • Bengaluru, Karnataka, India Observe Full time

    Observe.AI: Boosting Contact Center Performance with Live Conversation IntelligenceAbout UsWe're the fastest way to enhance contact center performance, leveraging our cutting-edge AI engine. With Observe.AI, you'll uncover valuable insights from 100% of customer interactions and optimize frontline team performance through coaching and end-to-end workflow...


  • Bengaluru, Karnataka, India CygnusPro Software Solutions Pvt. Ltd Full time

    Estimated salary: ₹30,00,000 - ₹40,00,000 per annumAbout CygnusPro Software Solutions Pvt. Ltd.We are a leading software solutions company that specializes in providing innovative data engineering services to our clients. Our team of experts is dedicated to delivering high-quality solutions that meet the evolving needs of businesses.Job Description:Data...


  • Bengaluru, Karnataka, India McLaren Strategic Solutions (MSS) Full time

    Unlock Your Potential as a Data Engineer with McLaren Strategic SolutionsAre you passionate about building efficient, large-scale data pipelines using distributed technologies like Hadoop and Spark? Do you have strong software engineering experience in Python, Scala, Java or equivalent?We are seeking an experienced Data Engineer to join our team at McLaren...


  • Bengaluru, Karnataka, India ADCI HYD 13 SEZ Full time

    About UsADCI HYD 13 SEZ is a leading provider of innovative data solutions, enabling teams across Amazon to effectively manage and analyze massive datasets. Our mission is to empower businesses with cutting-edge technologies that simplify complex data challenges.Job DescriptionWe are seeking an experienced Sr. Software Development Engineer to join our team...


  • Bengaluru, Karnataka, India NTT DATA Full time

    About the RoleWe are seeking a highly skilled Snowflake Data Engineer and Architect to join our team in Bangalore, India. This is a senior-level position that requires expertise in designing, developing, and maintaining large-scale data warehousing solutions using Snowflake.Key ResponsibilitiesDesign and implement efficient data models and architectures for...


  • Bengaluru, Karnataka, India PureSoftware Pvt Ltd. Full time

    Company OverviewPureSoftware Pvt Ltd. is a leading technology company that empowers businesses to drive innovation and growth through cutting-edge solutions.Job DescriptionWe are seeking an experienced Data Engineer to join our team. As a key member of our data engineering group, you will be responsible for designing, developing, and delivering large-scale...


  • Bengaluru, Karnataka, India Talent500 Full time

    Talent500 Seeks Data Engineer for Large-Scale AnalyticsEstimated salary range: $150,000 - $200,000 per year.About the Role:We are seeking an experienced Data Engineer to join our team at Talent500. As a key member of our analytics department, you will be responsible for designing and developing large-scale analytics solutions using Snowflake, dbt Cloud, and...


  • Bengaluru, Karnataka, India LinkedIn Full time

    At LinkedIn, we're pioneering a next-generation Tracking platform to efficiently collect interaction data for our billion-plus users. As a Staff Software Engineer on our Systems Infrastructure team, you'll play a crucial role in designing, implementing, and optimizing the performance of large-scale distributed systems with security and compliance in...

  • Data Engineer

    2 weeks ago


    Bengaluru, Karnataka, India CodeVyasa Full time

    Looking for a Senior Data Engineer to join our team in Bangalore. The ideal candidate should have at least 2+ years of experience in large-scale data processing and data pipeline development.We are a multinational software company, CodeVyasa, that is growing rapidly with offices in Florida and New Delhi. Our clientele spans across the US, Australia, and...


  • Bengaluru, Karnataka, India NTT DATA Full time

    About the RoleWe are seeking a highly skilled Senior Time Series Data Engineer to join our team in Bangalore, India. As a Senior Time Series Data Engineer, you will be responsible for designing, developing, and implementing large-scale data processing systems that handle complex time series data.Key ResponsibilitiesDesign and develop scalable data pipelines...


  • Bengaluru, Karnataka, India Infoblox Full time

    About the RoleAs a Staff Data Engineer at Infoblox, you will play a key role in developing platforms and products for our SaaS product line, delivering next-level networking for our customers.This is an exciting opportunity to work closely with data scientists and product teams to curate and refine data powering our latest cloud products.Key...


  • Bengaluru, Karnataka, India NTT DATA Services Full time

    Job DescriptionWe are seeking an experienced KDB developer to join our team in Bangalore, India. As a key member of our development team, you will be responsible for designing and implementing large-scale systems using KDB+, Linux, Shell, and CI/CD tools.About the Role:You will work on developing complex algorithms and engineering large systems with...


  • Bengaluru, Karnataka, India Adobe Full time

    Unlock the Power of Big DataWe're seeking a highly skilled Data Architect to join our team and lead the development of large-scale analytics platforms. With Adobe, you'll have the opportunity to work with cutting-edge technologies and collaborate with talented professionals to design and deliver exceptional digital experiences.About the RoleAs a Data...


  • Bengaluru, Karnataka, India Techs to Suit Inc Full time

    About Techs to Suit IncWe are a renowned mid-sized IT company based in Bangalore, India.Job Summary:We seek an experienced Data Engineering Architect to lead our large scale analytics projects. As a key member of our team, you will be responsible for architecting big data environments, leveraging appropriate visualization tools and rigorous hypothesis...

  • Senior Data Architect

    3 weeks ago


    Bengaluru, Karnataka, India NTT DATA Services Full time

    About the RoleNTT DATA Services is seeking a highly skilled Senior Data Architect to join our team in Bangalore, India.The Ideal CandidateThis is an exceptional opportunity for an experienced data professional to work with cutting-edge technologies and contribute to the design and development of innovative solutions. To be successful in this role, you will...


  • Bengaluru, Karnataka, India NETSACH GLOBAL Full time

    About the RoleWe are seeking an experienced Data Architect to join our team at Netsach Global, a leading Cyber Security Company. The ideal candidate will have a strong background in designing and developing large-scale data processing systems and architecture.Key ResponsibilitiesDesign, implement, and manage big data infrastructure and architecture to ensure...


  • Bengaluru, Karnataka, India Altimetrik Full time

    About the RoleWe are seeking a skilled Data Engineer to join our team at Altimetrik in Bangalore. As a Data Engineer, you will play a critical role in designing and developing large-scale data warehousing solutions.ResponsibilitiesDesign and develop ETL jobs across multiple platforms using AWS services like S3, Redshift, EMR, Lambda, Athena, and Glue.Develop...


  • Bengaluru, Karnataka, India ANSR Full time

    Job Overview">This role is responsible for designing and developing large-scale analytics solutions using data engineering principles. The ideal candidate will have a strong understanding of data architecture, solution design, and development best practices.