Big Data Specialist

2 months ago

india Deloitte Full time

We are looking for Candidates who are having experience as Big Data.


Experience Required: 10+ years.

Location: PAN India


Responsibilities:

  • Design, develop and deploy solutions using different tools, design principles and conventions.
  • Assemble large, complex data sets that meet functional / non-functional business requirements.
  • Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
  • Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources and ‘big data’ technologies.
  • Optimize data ingestion and processing performance by tuning configuration parameters, optimizing SQL queries, and implementing parallel processing techniques.
  • Monitor pipeline performance and resource utilization to identify bottlenecks and optimize system resources allocation.
  • Set up monitoring and alerting systems to track pipeline health, detect anomalies, and notify stakeholders of potential issues or failures.
  • Configure logging and metrics collection to facilitate troubleshooting and performance analysis.
  • Understand existing processes and facilitate change requirements as part of a structured change control process.
  • Maintain proper documentation for the solutions, test procedures and scenarios during UAT and Production phase.
  • Understand and implement Pipeline Development, Data Quality and Governance, Monitoring and Alerting, Data Virtualization and Query Optimization, Data Source Integration, Data Catalog Management, Security and Access Control, Integration with Applications and Ecosystems, Data Ingestion and Management
  • Coordinate with process owners and business to understand the as-is process and design the automation process flow.
  • Produce high-quality code resulting from knowledge of the latest frameworks, code peer review, and automated unit test scripts.
  • Collaborate with other developers and team members.

  • Big Data Architect

    7 months ago

    India Princeton IT Services Full time

    **Job Title**: Big Data Architect **Location**:Kakinada (work from office) or Bangalore (work from office) **Shift**: UK shift preferred **Job Overview**: We are seeking a highly skilled and motivated Big Data Architect to join our team. This individual will play a crucial role in designing and implementing scalable and efficient big data solutions,...

  • Big Data Developer

    4 weeks ago

    india Amossys Consulting Services Full time

    Position Title: Data Engineer Experience Required: 5+ years Employment Type: Full-time Employment Mode: Hybrid Location: Bangalore, Pune, Trivandrum. Industry: Banking & Insurance sector Job Summary: We are looking for a skilled Data Engineer with hands-on experience in PySpark and Big Data platforms to join our growing team. The ideal candidate...

  • Big Data Developer

    4 weeks ago

    india Amossys Consulting Services Full time

    Position Title: Data Engineer Experience Required: 5+ years Employment Type: Full-time Employment Mode: Hybrid Location: Bangalore, Pune, Trivandrum. Industry: Banking & Insurance sector Job Summary: We are looking for a skilled Data Engineer with hands-on experience in PySpark and Big Data platforms to join our growing team. The ideal candidate...

  • Big data developer

    3 days ago

    India Manuh Technologies Full time

    Big Data Engineer with 5+ years of experience on technologies Cloudera, Hadoop, sqoop, Spark, Py Spark, Impala, SQL Previous experince into big data projects using hadoop and spark. Atleast one project experience into Cloudera hadoop with Impala and Hue Strong Programming experience in Python and Py Spark Hands on experience in SQL using Impala ...

  • Big Data Engineer

    2 months ago

    india RiDiK (a Subsidiary of CLPS. Nasdaq: CLPS) Full time

    Job Summary:We are seeking a skilled and innovative Big Data Engineer to join our dynamic team. The ideal candidate will be responsible for designing, building, and maintaining scalable data processing systems. You will work with large datasets and implement data pipelines to ensure data quality and availability for analytics and business intelligence...

  • Big Data Developer

    4 days ago

    India Manuh Technologies Full time

    Big Data Engineer with 5+ years of experience on technologies Cloudera, Hadoop, sqoop, Spark, PySpark, Impala, SQLPrevious experince into big data projects using hadoop and spark.Atleast one project experience into Cloudera hadoop with Impala and HueStrong Programming experience in Python and PySparkHands on experience in SQL using ImpalaExperience in...

  • Big Data Developer

    2 days ago

    India Manuh Technologies Full time

    Big Data Engineer with 5+ years of experience on technologies Cloudera, Hadoop, sqoop, Spark, PySpark, Impala, SQL Previous experince into big data projects using hadoop and spark. Atleast one project experience into Cloudera hadoop with Impala and Hue Strong Programming experience in Python and PySpark Hands on experience in SQL using Impala Experience in...

  • Big Data Developer

    5 days ago

    india Manuh Technologies Full time

    Big Data Engineer with 5+ years of experience on technologies Cloudera, Hadoop, sqoop, Spark, PySpark, Impala, SQLPrevious experince into big data projects using hadoop and spark.Atleast one project experience into Cloudera hadoop with Impala and HueStrong Programming experience in Python and PySparkHands on experience in SQL using ImpalaExperience in...

  • Big Data Developer

    1 month ago

    india Tata Consultancy Services Full time

    Greetings from TCS!! Role : BigData Developer Experience range : 4 to 9 Years Location : Hyderabad/ Pune Skills Required : Bigdata –Scala/Java, Spark, Hive, Unix Shell Scripting Must Have : Hands on development experience in programming languages such as JAVA,SCALA using Maven, Apache Spark Frameworks and Unix Shell scripting Should be comfortable with...

  • Big data developer

    3 weeks ago

    India Luxoft Full time

    During the 2008 financial crisis, many big banks failed or faced issues due to liquidity issues. Lack of liquidity can kill any financial institution over the night. That's why it's so critical to constantly monitor liquidity risks and properly maintain collaterals. We are looking for a number of talented developers, who would like to join our...

  • Big Data Infrastructure Specialist

    3 weeks ago

    Metros/Anywhere in India/Multiple Locations Dotflick Solutions Full time

    Job SummaryWe are seeking a highly skilled Data Engineering Expert to join our team at Dotflick Solutions. The ideal candidate will have a strong background in big data infrastructure and experience with Hadoop clusters.Key ResponsibilitiesDesign and implement data ingestion pipelines between legacy data warehouses and big data stacksDevelop and deploy data...

  • Big Data Engineer

    3 weeks ago

    india Manuh Technologies Full time

    Position - Big Data EngineerLocation - Pune/Hyderabad/AnyExperience - 5+ Job SummaryAs a Big data engineer responsibilities include - review and analysis of requirements, design workshop participation and facilitation, design creation and documentation, data pipeline implementation, testing, bug fixing, release support and post release aftercare.You will be...

  • Big Data Engineer

    3 weeks ago

    India Manuh Technologies Full time

    Position - Big Data Engineer Location - Pune/Hyderabad/Any Experience - 5+ Job Summary As a Big data engineer responsibilities include - review and analysis of requirements, design workshop participation and facilitation, design creation and documentation, data pipeline implementation, testing, bug fixing, release support and post release aftercare. You...

  • Big data engineer

    3 weeks ago

    India Manuh Technologies Full time

    Position - Big Data Engineer Location - Pune/Hyderabad/Any Experience - 5+ Job Summary As a Big data engineer responsibilities include - review and analysis of requirements, design workshop participation and facilitation, design creation and documentation, data pipeline implementation, testing, bug fixing, release support and post release aftercare. ...

  • Big Data Architect

    4 weeks ago

    india Affine Full time

    Primary Role:Resource and Project Timeline EstimationDefining project and program management processPrioritizing task within project: Task allocation across resources – capable of evaluating resources by their skills and improvement areas - prioritizes resource developmentContributes to org-level activities such as sales collaterals, designing proposals,...

  • Big Data Architect

    4 weeks ago

    india Affine Full time

    Primary Role : Resource and Project Timeline Estimation Defining project and program management process Prioritizing task within project: Task allocation across resources – capable of evaluating resources by their skills and improvement areas - prioritizes resource development Contributes to org-level activities such as sales collaterals, designing...

  • Big Data Architect

    5 months ago

    India Persistent Systems Full time

    **About Persistent** We are a trusted Digital Engineering and Enterprise Modernization partner, combining deep technical expertise and industry experience to help our clients anticipate what’s next. Our offerings and proven solutions create a unique competitive advantage for our clients by giving them the power to see beyond and rise above. We work with...

  • Big Data Engineer

    4 weeks ago

    india Manuh Technologies Full time

    Position - Big Data Engineer Location - Pune/Hyderabad/Any Experience - 5+ Job Summary As a Big data engineer responsibilities include - review and analysis of requirements, design workshop participation and facilitation, design creation and documentation, data pipeline implementation, testing, bug fixing, release support and post release aftercare. You...

  • Big Data Engineer

    1 month ago

    India Manuh Technologies Full time

    Position - Big Data EngineerLocation - Pune/Hyderabad/AnyExperience - 5+ Job SummaryAs a Big data engineer responsibilities include - review and analysis of requirements, design workshop participation and facilitation, design creation and documentation, data pipeline implementation, testing, bug fixing, release support and post release aftercare.You will be...

  • Big Data Engineer

    1 month ago

    india Manuh Technologies Full time

    Position - Big Data EngineerLocation - Pune/Hyderabad/AnyExperience - 5+ Job SummaryAs a Big data engineer responsibilities include - review and analysis of requirements, design workshop participation and facilitation, design creation and documentation, data pipeline implementation, testing, bug fixing, release support and post release aftercare.You will be...