Big Data Engineer

2 weeks ago

hyderabad, India ADP Full time
Job Description: Big Data Application Developer
Experience Required: 3-5 years
We are seeking a highly skilled big data Developer with expertise in Python, Scala, and Java to join our dynamic team. The successful candidate will be responsible for designing, developing, and implementing scalable business applications. This role requires a deep understanding of microservices architecture, design patterns, and AWS integrations. The ideal candidate will be adept at conducting option analysis, designing new modules, and delivering measurable outcomes. This position involves working closely with cross-functional teams to deliver innovative solutions that meet and exceed business objectives.
Key Responsibilities:
Design and Development: Work on design, development, and implementation of new business application modules using PySpark, Python, Scala, and Java, ensuring they are robust, scalable, and maintainable.
Option Analysis: Conduct thorough option analysis for technical solutions, considering multiple approaches and selecting the most effective and efficient one.
Microservices Architecture: Design and develop applications following microservices architecture principles, ensuring modularity, scalability, and ease of maintenance.
AWS & Databricks Integration: Implement seamless integration between AWS services and Databricks, optimizing for performance and reliability.
Design Patterns: Apply industry-standard design patterns to ensure code quality, maintainability, and scalability of applications.
Testing & Quality Assurance: Develop and execute unit and integration tests using Pytest and other frameworks, ensuring high-quality code that meets business requirements.
Containerization: Use Docker to containerize applications, facilitating consistent and reliable deployments across different environments.
Collaboration and Communication: Collaborate with business analysts, product managers, and other developers to gather requirements, design solutions, and present outcomes to stakeholders.
Performance Monitoring: Monitor application performance in production environments and implement necessary optimizations to maintain high performance and reliability.
Security Best Practices: Implement security best practices throughout the application development lifecycle, including data encryption, authentication, and authorization.
Mentorship and Leadership: Provide guidance and mentorship to junior developers, fostering a collaborative and learning-driven environment.
Adaptability: Stay up-to-date with industry trends, continuously improving skills and adopting new tools, technologies, and methodologies as needed.
Qualifications:
Education: Bachelor’s degree in computer science, Information Technology, or a related field.
Experience: 3-5 years of professional experience in software development with a focus on PySpark, Python, Scala, and Java.
Technical Skills:
Expertise in PySpark, Python, Scala, and Java.
Strong knowledge of microservices architecture and design patterns.
Experience with AWS services and Databricks integration.
Proficiency in writing tests using Pytest and implementing CI/CD pipelines.
Hands-on experience with Docker for containerization.
Knowledge of security best practices in software development.
Soft Skills:
Strong analytical and problem-solving skills.
Excellent communication and collaboration abilities.
Ability to work independently and as part of a team in a dynamic environment.

  • Big Data Engineer

    4 weeks ago

    Hyderabad, India NucleusTeq Full time

    Big Data Engineer Responsibilities: Big Data Engineer with 5-6 years of experienceExperience in leading a team of engineers and a good attitude toward learning the domain and implementationStrong exposure and expertise in Spark (Primary), Scala/Java (Scala Primary), Airflow Orchestration and AWS.Finalizing the scope of the system and delivering...

  • Big Data Engineer

    1 week ago

    Hyderabad, India NucleusTeq Full time

    Big Data Engineer Responsibilities: Big Data Engineer with 5+ years of experienceExperience in leading a team of engineers and a good attitude toward learning the domain and implementationStrong exposure and expertise in Spark (Primary), Scala/Java (Scala Primary), Airflow Orchestration and AWS.Finalizing the scope of the system and delivering...

  • Big data engineer

    3 months ago

    Hyderabad, India Virtusa Full time

    Big data engineer - CREQ189284 Description Position: Big data engineer Primary: big data concepts, aws, pyspark Location: HYD Create Trigger based automation framework for Data Migration Identify roles/access needed for data migration from federated bucket to managed bucket and Build APIs for the same Integrate CDMS framework with Lake and Data bridge...

  • Big Data Engineer

    4 months ago

    Hyderabad, India TechSophy Full time

    Job Description: Roles&Responsibilities : 1. As an Engineer, you are responsible to analyze large data sets to identify errors in the map. 2. As an Engineer, you are responsible to design and implement a complex algorithm for resolving the issue, review the solution with a team of engineers and analysts, and integrate the resulting solution into...

  • Big Data Python Engineer

    3 months ago

    Hyderabad, India Virtusa Full time

    Big Data Python Engineer - CREQ191701 Description Role GDT WS Big Data Python Engineer Exp 5+ Location Pune/Hyderabad MUST Have Skills Design and implement data processing pipelines using Python + SCALA programming and big data technologies such as Hadoop, MapReduce, Hive, and Spark. Airflow Jenkins Understand business requirements to code up data assets...

  • AWS Big Data Engineer

    3 months ago

    Hyderabad, India Tiger Analytics Full time

    Job Description We are looking for a Senior Data Engineer to be based out of our Chennai, Bangalore & Hyderabad offices. This role involves a combination of hands-on contribution, customer engagement, and technical team management. As a Senior Data Engineer, you will Design and build solutions for near real-time stream processing as well as batch...

  • Big Data Engineer

    2 weeks ago

    hyderabad, India Anblicks Full time

    4-6 years of experience:Experience in hadoop/big data technologies.experience in spark.experience in Snowflakeexperience working on Google or AWS cloud developing data solutions. Certifications preferred.Hands-on experience with Python/Pyspark/Scala and basic libraries for machine learning is required;Experience with containerization and related technologies...

  • GCP Big Data Engineer

    2 months ago

    Hyderabad, India Tetrahed INC Full time

    Relevant experience for engagement: 10Years Relevant skillset: Maintain architecture principles guidelinesand standards Data Warehousing Programming Language: Python/Java Big Data DataAnalytics GCP ServicesRelevanttechnology domains: Experience in designing& implementing solution in mentioned areas: Strong GoogleCloud Platform Data Components BigQuery...

  • Big Data Engineer

    3 weeks ago

    Hyderabad, India Anblicks Full time

    4-6 years of experience:Experience in hadoop/big data technologies.experience in spark.experience in Snowflakeexperience working on Google or AWS cloud developing data solutions. Certifications preferred.Hands-on experience with Python/Pyspark/Scala and basic libraries for machine learning is required;Experience with containerization and related technologies...

  • Big Data Engineer

    2 weeks ago

    Hyderabad, India Anblicks Full time

    4-6 years of experience: Experience in hadoop/big data technologies. experience in spark. experience in Snowflake experience working on Google or AWS cloud developing data solutions. Certifications preferred. Hands-on experience with Python/Pyspark/Scala and basic libraries for machine learning is required; Experience with containerization and related...

  • Big Data Engineer, Data Modeling

    4 months ago

    Hyderabad, India data.ai Remote Work Freelance Full time

    What can you tell your friends when they ask you what you do? We’re looking for an experienced Big Data Engineer who can create innovative new products in the analytics and data space. You will participate in the development that creates the world's #1 mobile app analytics service. Together with the team, you will build out new product features and...

  • Senior Big Data Engineer

    1 day ago

    Hyderabad, India Findernest Software Services Pvt Ltd Full time

    Overview:The SeniorBig Data Engineer plays a crucial role in our organizationresponsible for designing developing and maintaining scalable bigdata solutions. This role is essential in harnessing the power ofdata to drive informed business decisions and strategicinitiatives.KeyResponsibilities:Design andimplement big data solutions using technologies such as...

  • Senior Big Data Software Engineer

    4 months ago

    Hyderabad, India Epam Full time

    Description EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that...

  • Big Data Analyst

    3 months ago

    Hyderabad, India Randstad Enterprise Full time

    Shift : 2pm-11am ISTWork Mode : Remotewe areRandstad Sourceright’s global talent solutions provides instant access to experienced recruitment and contingent workforce management support by combining technology, analytics and deep global and local expertise. Our operations consist of the client aligned service delivery teams operating across RPO, MSP and...

  • Technoprism - Data Engineer - Big Data

    1 month ago

    Hyderabad, India TECNOPRISM PRIVATE LIMITED Full time

    Job Title : Data EngineerLocation : Hyderabad, IndiaExperience : 5-7 yearsAbout the Role : We are seeking a highly skilled and experienced Data Engineer to join our dynamic team in Hyderabad. The ideal candidate will have a solid background in data engineering, with at least 5 years of experience in supply chain analytics. As a Data Engineer, you will play a...

  • Senior Big data Devops Engineer

    4 months ago

    Hyderabad, India ServiceNow Full time

    Job DescriptionThe Big Data team plays a critical and strategic role in ensuring that ServiceNow can exceed the availability and performance SLAs of the ServiceNow Platform powered Customer instances - deployed across the ServiceNow cloud and Azure cloud.  Our mission is to:Deliver state of the art Monitoring, Analytics and Actionable Business Insights by...

  • Senior Big data Devops Engineer

    4 months ago

    Hyderabad, India ServiceNow Full time

    Company DescriptionAt ServiceNow, our technology makes the world work for everyone, and our people make it possible. We move fast because the world can’t wait, and we innovate in ways no one else can for our customers and communities. By joining ServiceNow, you are part of an ambitious team of change makers who have a restless curiosity and a drive for...

  • Senior Big data Devops Engineer

    4 months ago

    Hyderabad, Telangana, India ServiceNow Full time

    Company DescriptionAt ServiceNow, our technology makes the world work for everyone, and our people make it possible. We move fast because the world can't wait, and we innovate in ways no one else can for our customers and communities. By joining ServiceNow, you are part of an ambitious team of change makers who have a restless curiosity and a drive for...

  • Big Data Analyst

    3 months ago

    hyderabad, India Randstad Enterprise Full time

    Shift : 2pm-11am IST Work Mode : Remote we are Randstad Sourceright’s global talent solutions provides instant access to experienced recruitment and contingent workforce management support by combining technology, analytics and deep global and local expertise. Our operations consist of the client aligned service delivery teams operating across RPO, MSP...

  • Big Data Analyst

    3 months ago

    Hyderabad, India Randstad Enterprise Full time

    Shift : 2pm-11am ISTWork Mode : Remotewe areRandstad Sourceright’s global talent solutions provides instant access to experienced recruitment and contingent workforce management support by combining technology, analytics and deep global and local expertise. Our operations consist of the client aligned service delivery teams operating across RPO, MSP and...