Sr Big Data Engineer

1 week ago

India Rackspace Full time
About the Role: We are seeking a highly skilled and experienced Senior Big Data Engineer to join our dynamic team. The ideal candidate will have a strong background in developing and scaling both stream and batch processing systems, and a solid understanding of public cloud technologies, especially GCP. This role involves working in a remote environment, requiring excellent communication skills and the ability to solve complex problems independently and creatively. What you will be doing Build a reusable, and reliable code for stream and batch processing systems at scale. This includes working with technologies like Pub/Sub, Kafka, Kinesis, DataFlow, Flink, Hadoop, Pig, Hive, and Spark. Implementing automation/DevOps best practices for CI/CD, IaC, Containerization, etc.

Requirements:

Job Title: Sr Big Data Engineer (GCP) - Airflow and Oozie About the Role: We are seeking a highly skilled and experienced Senior Big Data Engineer to join our dynamic team. The ideal candidate will have a strong background in developing batch processing systems, with extensive experience in Oozie, the Apache Hadoop ecosystem, Airflow, and a solid understanding of public cloud technologies, especially GCP. This role involves working in a remote environment, requiring excellent communication skills and the ability to solve complex problems independently and creatively. What you will be doing Develop scalable and robust code for batch processing systems. This includes working with technologies like Hadoop, Oozie, Pig, Hive, Map Reduce, Spark (Java), Python, Hbase Develop, Manage and optimize data workflows using Oozie and Airflow within the Apache Hadoop ecosystem Leverage GCP for scalable big data processing and storage solutions Implementing automation/DevOps best practices for CI/CD, IaC, etc. Requirements: Experience with GCP managed services and understanding of cloud-based batch processing systems are critical. Proficiency in Oozie, Airflow, Map Reduce, Java Strong programming skills with Java (specifically Spark), Python, Pig, and SQL Expertise in public cloud services, particularly in GCP. Proficiency in the Apache Hadoop ecosystem with Oozie, Pig, Hive, Map Reduce Familiarity with BigTable and Redis Experienced in Infrastructure and Applied DevOps principles in daily work. Utilize tools for continuous integration and continuous deployment (CI/CD), and Infrastructure as Code (IaC) like Terraform to automate and improve development and release processes. Ability to tackle complex challenges and devise effective solutions. Use critical thinking to approach problems from various angles and propose innovative solutions. Worked effectively in a remote setting, maintaining strong written and verbal communication skills. Collaborate with team members and stakeholders, ensuring clear understanding of technical requirements and project goals. Proven experience in engineering batch processing systems at scale. Hands-on experience in public cloud platforms, particularly GCP. Additional experience with other cloud technologies is advantageous.

Must Have:

Must Have Google Associate Cloud Engineer Certification or other Google Cloud Professional level certification 10+ years of experience in customer-facing software/technology or consulting 5+ years of experience with "on-premises to cloud" migrations or IT transformations 5+ years of experience building, and operating solutions built on GCP (ideally) or AWS/Azure Technical degree: Computer Science, software engineering or related About Rackspace Technology We are the multicloud solutions experts. We combine our expertise with the world's leading technologies — across applications, data and security — to deliver end-to-end solutions. We have a proven record of advising customers based on their business challenges, designing solutions that scale, building and managing those solutions, and optimizing returns into the future. Named a best place to work, year after year according to Fortune, Forbes and Glassdoor, we attract and develop world-class talent. Join us on our mission to embrace technology, empower customers and deliver the future. More on Rackspace Technology Though we're all different, Rackers thrive through our connection to a central goal: to be a valued member of a winning team on an inspiring mission. We bring our whole selves to work every day. And we embrace the notion that unique perspectives fuel innovation and enable us to best serve our customers and communities around the globe. We welcome you to apply today and want you to know that we are committed to offering equal employment opportunity without regard to age, color, disability, gender reassignment or identity or expression, genetic information, marital or civil partner status, pregnancy or maternity status, military or veteran status, nationality, ethnic or national origin, race, religion or belief, sexual orientation, or any legally protected characteristic. If you have a disability or special need that requires accommodation, please let us know.

  • Big Data Engineer

    3 weeks ago

    india Awign Expert Full time

    Job Description This is a remote position. Job Title: Big Data Solutions Engineer Reporting to: Director, Product Management Location: India (Remote) Our Ideal Candidate Our company is seeking a talented Big Data Solutions Engineer to join our team. The ideal candidate will be responsible for designing, developing, and...

  • Big Data Engineer

    1 week ago

    India Careerfit Full time

    Job Title: Big Data Solutions EngineerAbout the Company: Our company is a leading tech firm specializing in innovative data solutions that empower businesses to make informed decisions.Responsibilities :Solution Design : Design, develop, and maintain scalable big data solutions and data pipelines for efficient data ingestion, processing, storage, and...

  • Sr Big Data Engineer

    1 month ago

    india Rackspace Full time

    About the Role: We are seeking a highly skilled and experienced Senior Big Data Engineer to join our dynamic team. The ideal candidate will have a strong background in developing and scaling both stream and batch processing systems, and a solid understanding of public cloud technologies, especially GCP. This role involves working in a remote environment,...

  • Sr Big Data Engineer

    3 weeks ago

    india Rackspace Full time

    About the Role: We are seeking a highly skilled and experienced Senior Big Data Engineer to join our dynamic team. The ideal candidate will have a strong background in developing and scaling both stream and batch processing systems, and a solid understanding of public cloud technologies, especially GCP. This role involves working in a remote environment,...

  • Lead Big Data Engineer

    1 week ago

    India Grownex HR Solution Full time

    Role : Lead Big Data Engineer Work Mode : Remote(5 Days working) The Opportunity - Development team is establishing a new offshore product development team for its partner , that is building next-generation Big Data, Cloud-Based Business Operation Support technology for utilities, retail energy suppliers and Community Choice Aggregators (CCA). The candidate...

  • Big Data Solutions Engineer

    7 days ago

    India Gethyr Full time

    Our company is seeking a talented Big Data Solutions Engineer to join our team. The ideal candidate will be responsible for designing, developing, and implementing our proprietary Bigdata-powered Fin Crime solution for our clients. The candidate will work closely with the software development team, data scientists, and product managers to understand...

  • Big Data Engineer

    2 weeks ago

    india Evnek Technologies Pvt Ltd Full time

    Job Description This is a remote position. Duration:  6 MonthLocation:  RemoteTimings: Full Time (As per company timings)Notice Period: within 15 days or immediate joinerExperience: 4-6 Years   Responsibilities Create and maintain data pipelines that transfer data from select locations (maintaining security protocols) into our Big Data financial...

  • Big Data Engineer

    1 week ago

    India ATech Full time

    Designation:BIG DATA ENGINEERJob Description:Your Role and Responsibilities: Understand a data warehousing solution and able to work independently in such an environment Responsible in Project development and delivery experience of a few good size projects Design, build, optimize and support new and existing data models and ETL processes based on our clients...

  • Big Data Engineer

    1 week ago

    Anywhere in India/Multiple Locations D-TechWorks Pvt Ltd Full time

    We are hiring for Big Data AI/ML Engineer - Remote , India Position : Big Data AI/ML EngineerExperience : 6 to 9 YrsLocation : Remote We are seeking a talented Data Engineer with AI & ML knowledge to join our team. As Data Engineer or ML Ops Engineer, your primary responsibility will be to develop & integrate ML solutions that focus on technology...

  • Data Engineer

    3 weeks ago

    india Evnek Technologies Pvt Ltd Full time

    Job Description This is a remote position. Role: Data Engineer  Experience: 2+Years Notice period: Immediate Location: Anywhere from India Job Type: Full Time Responsibilities: Designing & building data architectures including the design & set-up of scalable (big) data lakes (full cloud or hybrid cloud), scheduled transformations &...

  • Data Engineer

    1 month ago

    india Evnek Technologies Pvt Ltd Full time

    Job Description This is a remote position. Role: Data Engineer  Experience: 2+Years Notice period: Immediate Location: Anywhere from India Job Type: Full Time Responsibilities: Designing & building data architectures including the design & set-up of scalable (big) data lakes (full cloud or hybrid cloud), scheduled transformations &...

  • Big Data Architect

    7 days ago

    India ARR Recruitment Solutions Full time

    About the Role : We are seeking a highly experienced Big Data Architect to join our team and lead the design, development, and implementation of our big data solutions. This role is ideal for a candidate with a deep understanding of big data technologies and a proven track record of delivering scalable and robust big data architectures.Responsibilities :...

  • Big Data Engineer

    4 days ago

    india Programming.com Full time

    Job Title : Big Data EngineerLocation : RemoteAbout Us :We are a leading specializing in software development . Our team is dedicated to leveraging cutting-edge technologies to deliver scalable and efficient solutions to our clients globally.Role Overview :We are seeking a highly skilled Big Data Engineer with a strong background in designing and developing...

  • GCP Big data Engineer

    1 week ago

    India IKAI Technology Solutions Full time

    We are seeking a talented Big Data + GCP Engineer to join our dynamic team. In this role, you will be responsible for designing, implementing, and maintaining scalable data pipelines and infrastructure on Google Cloud Platform (GCP). You will collaborate closely with cross-functional teams to ensure that our data platform meets the needs of our business...

  • GCP Big data Engineer

    1 week ago

    india IKAI Technology Solutions Full time

    We are seeking a talented Big Data + GCP Engineer to join our dynamic team. In this role, you will be responsible for designing, implementing, and maintaining scalable data pipelines and infrastructure on Google Cloud Platform (GCP). You will collaborate closely with cross-functional teams to ensure that our data platform meets the needs of our business...

  • GCP Big data Engineer

    1 week ago

    India IKAI Technology Solutions Full time

    We are seeking a talented Big Data + GCP Engineer to join our dynamic team. In this role, you will be responsible for designing, implementing, and maintaining scalable data pipelines and infrastructure on Google Cloud Platform (GCP). You will collaborate closely with cross-functional teams to ensure that our data platform meets the needs of our business...

  • Big Data Architect

    1 week ago

    India Swift Strategic Staff Solutions INC Full time

    Responsibilities :Data Ingestion & Processing : Design and build robust data pipelines using tools like Apache Spark, Flink, or Kafka to ingest, transform, and process massive datasets from various sources.Big Data Technologies : Utilize your expertise in big data frameworks like Hadoop, YARN, and HDFS to manage, store, and analyze large volumes of data...

  • Big Data Quality Assurance Engineer

    1 week ago

    India Leadway Resource Full time

    We are seeking a highly skilled and experienced Big Data QA Engineer to join our team. You will play a critical role in ensuring the quality of our big data platform and data solutions. You will leverage your expertise in Big Data testing methodologies, automation frameworks, and scripting languages to design, develop, and execute comprehensive test...

  • Lead Big Data/Data Engineer

    18 hours ago

    india TechTiv Software Solutions Full time

    The Opportunity : Digit88 development team is establishing a new offshore product development team for its partner , that is building next-generation Big Data, Cloud-Based Business Operation Support technology for utilities, retail energy suppliers and Community Choice Aggregators (CCA). The candidate would be joining an existing team of outstanding data...

  • Freelance Big Data Engineer

    1 week ago

    India Leading MNC Full time

    Looking for a Freelance Big Data Solutions Engineer to join a team of rockstar developers. The candidate should have a minimum of 4+experience. There are multiple openings. If you're looking for freelance/ part time opportunity (along with your day job) & a chance to work with the top 0.1% of developers in the industry, this one is for you You will report...