Big Data Engineer

1 month ago

Hyderabad, India TECHSTAR SOFTWARE DEVELOPMENT INDIA PVT LTD Full time
Job Title: Big Data Engineer

As a Big Data Engineer at TECHSTAR SOFTWARE DEVELOPMENT INDIA PVT LTD, you will be responsible for designing and developing big data solutions that drive business growth and innovation.

Key Responsibilities:
  • Design and develop big data pipelines for data ingestion and transformation using Java, Scala, or Python.
  • Partner with domain experts, product managers, analysts, and data scientists to develop Big Data pipelines in Hadoop.
  • Work with data scientists to build client pipelines using heterogeneous sources and provide engineering services for data science applications.
  • Ensure automation through CI/CD across platforms both in cloud and on-premises.
  • Define needs around maintainability, testability, performance, security, quality, and usability for the data platform.
  • Drive implementation, consistent patterns, reusable components, and coding standards for data engineering processes.
  • Convert SAS-based pipelines into languages like PySpark, and Scala to execute on Hadoop and non-Hadoop ecosystems.
  • Tune Big data applications on Hadoop and non-Hadoop platforms for optimal performance.
  • Apply an in-depth understanding of how data analytics collectively integrate within the sub-function as well as coordinate and contribute to the objectives of the entire function.
  • Produce a detailed analysis of issues where the best course of action is not evident from the information available, but actions must be recommended/taken.
  • Assess risk when business decisions are made, demonstrating particular consideration for the firm's reputation and safeguarding Citigroup, its clients, and assets, by driving compliance with applicable laws, rules, and regulations, adhering to Policy, applying sound ethical judgment regarding personal behavior, conduct, and business practices, and escalating, managing and reporting control issues with transparency.
Requirements:
  • 6+ years of total IT experience.
  • 4+ years of experience with Hadoop (Cloudera)/big data technologies.
  • Knowledge of the Hadoop ecosystem and Big Data technologies Hands-on experience with the Hadoop eco-system (HDFS, MapReduce, Hive, Pig, Impala, Spark, Kafka, Kudu, Solr).
  • Experience in designing and developing Data Pipelines for Data Ingestion or Transformation using Java Scala or Python.
  • Experience with Spark programming (Pyspark, Scala, or Java).
  • Hands-on experience with Python/Pyspark/Scala and basic libraries for machine learning is required.
  • Proficient in programming in Java or Python with prior Apache Beam/Spark experience a plus.
  • System level understanding - Data structures, algorithms, distributed storage & compute.
  • Can-do attitude on solving complex business problems, good interpersonal and teamwork skills.

  • Big Data Engineer

    3 weeks ago

    Hyderabad, Telangana, India data Full time

    Overview: We're seeking a skilled Big Data Engineer to join our team at data.ai. As a key member of our engineering team, you will be responsible for designing and implementing innovative data solutions that drive business growth. If you're passionate about working with large data sets, have experience with big data technologies, and are eager to...

  • Big Data Engineer, Data Modeling Expert

    4 weeks ago

    Hyderabad, Telangana, India data Full time

    Unlock the Power of Data with UsWe're seeking an experienced Big Data Engineer to join our team and contribute to the development of innovative analytics and data products. As a key member of our engineering team, you will work closely with Data Scientists, Data Engineers, Product Managers, and Software Architects to design and implement complex data product...

  • Big Data Systems Engineer

    3 weeks ago

    Hyderabad, Telangana, India NTT DATA Full time

    Job Title: Data EngineerJob Summary:We are seeking a skilled Data Engineer to design, build, and maintain large-scale data systems. The ideal candidate will have expertise in big data ecosystems, data processing, and data storage.Key Responsibilities:Design and develop scalable data systems for business applications.Work closely with data scientists to...

  • Big Data Engineer

    3 weeks ago

    Hyderabad, Telangana, India NTT DATA Services Full time

    Data Engineer RoleData engineers at NTT DATA Services are responsible for building and testing scalable Big Data ecosystems for our clients. They work closely with data scientists to ensure the source data is aggregated and cleansed, and with product managers to understand the business objectives. Data engineers must have expertise in all phases of data...

  • Big Data Engineer

    3 weeks ago

    Hyderabad, Telangana, India ANB IT Solutions Pvt Ltd Full time

    Key ResponsibilitiesCollaborate with cross-functional teams to design, develop, and maintain big data solutions using Google Cloud Big Query. Key responsibilities include data ingestion, transformation, and analytics.Data Engineering and DevelopmentDevelop and maintain ETL/ELT processes to ensure data integrity and consistency. Utilize Google Cloud Composer,...

  • Big Data Engineer

    4 weeks ago

    Hyderabad, Telangana, India Virtusa Full time

    Job Title: Big Data EngineerJob Summary: We are seeking a skilled Big Data Engineer to design and implement data processing pipelines using Python and big data technologies such as Hadoop, MapReduce, Hive, and Spark.Key Responsibilities:Design and implement data processing pipelines using Python and big data technologies.Develop and maintain data assets...

  • Big Data Engineer

    4 weeks ago

    Hyderabad, Telangana, India Vidpro Consultancy Services Full time

    About the Role:Vidpro Consultancy Services is seeking a highly skilled Big Data Engineer to join our team. As a key member of our data architecture team, you will be responsible for designing and implementing large-scale Big Data architectures using Java and Python.Key Responsibilities:Design and implement Big Data architectures using Hadoop and SparkDevelop...

  • Big Data Engineer

    3 months ago

    Hyderabad, India NucleusTeq Full time

    Big Data Engineer Responsibilities: Big Data Engineer with 5+ years of experienceExperience in leading a team of engineers and a good attitude toward learning the domain and implementationStrong exposure and expertise in Spark (Primary), Scala/Java (Scala Primary), Airflow Orchestration and AWS.Finalizing the scope of the system and delivering...

  • Big Data Engineer

    1 month ago

    Hyderabad, Telangana, India TechSophy Full time

    Job Title: Big Data EngineerJob Summary:We are seeking a highly skilled Big Data Engineer to join our team at TechSophy. As a Big Data Engineer, you will be responsible for designing, developing, and maintaining large-scale data processing systems.Key Responsibilities:Analyze large data sets to identify errors in the map.Design and implement complex...

  • Big data engineer

    5 months ago

    Hyderabad, India Virtusa Full time

    Big data engineer - CREQ189284 Description Position: Big data engineer Primary: big data concepts, aws, pyspark Location: HYD Create Trigger based automation framework for Data Migration Identify roles/access needed for data migration from federated bucket to managed bucket and Build APIs for the same Integrate CDMS framework with Lake and Data bridge...

  • Big Data Engineer

    1 month ago

    hyderabad, India RiDiK (a Subsidiary of CLPS. Nasdaq: CLPS) Full time

    Job Summary:We are seeking a skilled and innovative Big Data Engineer to join our dynamic team. The ideal candidate will be responsible for designing, building, and maintaining scalable data processing systems. You will work with large datasets and implement data pipelines to ensure data quality and availability for analytics and business intelligence...

  • Big data engineer

    1 month ago

    Hyderabad, India RiDiK Full time

    Job Summary: We are seeking a skilled and innovative Big Data Engineer to join our dynamic team. The ideal candidate will be responsible for designing, building, and maintaining scalable data processing systems. You will work with large datasets and implement data pipelines to ensure data quality and availability for analytics and business intelligence...

  • Big Data Engineer

    1 month ago

    Hyderabad, Telangana, India RiDiK (a Subsidiary of CLPS. Nasdaq: CLPS) Full time

    Job SummaryWe are seeking a highly skilled and innovative Big Data Engineer to join our dynamic team at RiDiK (a Subsidiary of CLPS. Nasdaq: CLPS). The ideal candidate will be responsible for designing, building, and maintaining scalable data processing systems that support analytics and machine learning initiatives.Key ResponsibilitiesDesign and implement...

  • Senior Big Data Engineer

    4 weeks ago

    Hyderabad, Telangana, India Vidpro Consultancy Services Full time

    Job Title: Senior Big Data EngineerJob Summary: Vidpro Consultancy Services is seeking a highly skilled Senior Big Data Engineer to join our team. As a key member of our engineering team, you will be responsible for designing, developing, and maintaining large-scale big data architectures using Java, Python, and Hadoop technologies.Responsibilities:Design...

  • Big Data Engineer

    4 weeks ago

    Hyderabad, Telangana, India Nisum Full time

    About the RoleWe are seeking a highly skilled Big Data Engineer to join our team at Nisum. As a Big Data Engineer, you will be responsible for designing, developing, and maintaining large-scale data processing systems.Key ResponsibilitiesAnalyze complex business requirements and generate technical specifications to design or redesign complex software...

  • Big Data Engineer

    4 weeks ago

    Hyderabad, Telangana, India Devenir Software Private Limited Full time

    Job DescriptionWe are seeking a skilled Big Data Engineer to join our team at Devenir Software Private Limited. The ideal candidate will have hands-on experience with Big Data Hadoop, Spark, and AWS Cloud technologies.Key Responsibilities:Design and develop scalable data pipelines using Big Data Hadoop and Spark.Implement data storage solutions using AWS S3,...

  • Data Scientist

    2 days ago

    Hyderabad, Telangana, India Virtusa Full time

    About the Role">Virtusa is seeking an experienced Data Scientist to join our team as a Big Data Engineer. This role will involve designing and implementing data processing pipelines using Python and big data technologies such as Hadoop, MapReduce, Hive, and Spark.">Responsibilities">">Design and implement data processing pipelines using Python programming...

  • Big Data Engineer

    2 months ago

    Hyderabad, India RiDiK (a Subsidiary of CLPS. Nasdaq: CLPS) Full time

    Job Summary:We are seeking a skilled and innovative Big Data Engineer to join our dynamic team. The ideal candidate will be responsible for designing, building, and maintaining scalable data processing systems. You will work with large datasets and implement data pipelines to ensure data quality and availability for analytics and business intelligence...

  • Data Engineer for Big Data Platform

    3 weeks ago

    Hyderabad, Telangana, India DBS Bank Full time

    Data Engineer for Big Data PlatformWe are looking for a Data Engineer to join our team and work on our Big Data platform. As a Data Engineer, you will be responsible for designing, developing, and maintaining our data architecture, ensuring data quality, and optimizing data processing pipelines.Key Responsibilities:Design and develop data pipelines using...

  • Senior Big Data Engineer

    4 weeks ago

    Hyderabad, Telangana, India Findernest Software Services Pvt Ltd Full time

    About the RoleWe are seeking a highly skilled Senior Big Data Engineer to join our team at Findernest Software Services Pvt Ltd. As a Senior Big Data Engineer, you will be responsible for designing, developing, and maintaining scalable big data solutions that drive business growth and decision-making.Key ResponsibilitiesDesign and implement big data...