Big Data Developer

6 days ago

India Luxoft Full time

Project Description:

As a Senior Engineer in the Data Engineering & Analytics team, you will develop data & analytics solutions that sit atop vast datasets gathered by retail stores, restaurants, banks, and other consumer-focused companies. The challenge will be to create high-performance algorithms, cutting-edge analytical techniques including machine learning and artificial intelligence, and intuitive workflows that allow our users to derive insights from big data that in turn drive their businesses. You will have the opportunity to create high-performance analytic solutions based on data sets measured in the billions of transactions and front-end visualizations to unleash the value of big data. You will have the opportunity to develop data-driven innovative analytical solutions and identify opportunities to support business and client needs in a quantitative manner and facilitate informed recommendations/decisions through activities like building ML models, automated data pipelines, designing data architecture/schema, performing jobs in big data cluster by using different execution engines and program languages such as Hive/Impala, Python, Java, Kafka, Spark, R, etc.


Responsibilities:

  • Hands-on developer who writes good quality, secure code that is modular, functional, and testable.
  • Drive the evolution of Data & Services products/platforms with an impact-focused on data science and engineering.
  • Design and implement scalable data architecture and data pipelines
  • Solving complex problems with multi-layered data sets, as well as optimizing existing machine learning libraries and frameworks.
  • Provide support for deployed data applications and analytical models by being a trusted advisor to Data Scientists and other data consumers by identifying data problems and guiding issue resolution with partner Data Engineers and source data providers.
  • Ensure proper data governance policies are followed by implementing or validating Data Lineage, Quality checks, classification, etc.
  • Discover, ingest, and incorporate new sources of real-time, streaming, batch, and API-based data into our platform to enhance the insights we get from running tests and expand the ways and properties on which we can test Experiment with new tools to streamline the development, testing, deployment, and running of our data pipelines.
  • Participate in the development of data and analytic infrastructure for product development
  • Continuously innovate and determine new approaches, tools, techniques & technologies to solve business problems and generate business insights & recommendations
  • Partner with roles across the organization including consultants, engineering, and sales to determine the highest priority problems to solve
  • Evaluate trade-offs between many possible analytics solutions to a problem, taking into account usability, technical feasibility, timelines, and differing stakeholder opinions to make a decision
  • Break large solutions into smaller, releasable milestones to collect data and feedback from product managers, clients, and other stakeholders
  • • Evangelize releases to users, incorporating feedback, and tracking usage to inform future development
  • Work with small, cross-functional teams to define the vision, establish team culture and processes
  • Consistently focus on key drivers of organization value and prioritize operational activities accordingly
  • Escalate technical errors or bugs detected in project work
  • Maintain awareness of relevant technical and product trends through self-learning/study, training classes, and job shadowing.
  • Support the building of scaled machine learning production systems by designing pipelines and engineering infrastructure.


Mandatory Skills Description:

  • Working proficiency in using Python/Scala, Spark (tuning jobs), SQL, Hadoop platforms to build Big Data products & platforms.
  • Good programming skills in Java and spring boot and Junit.
  • Knowledge in software development test approaches & frameworks
  • Familiarity with RESTful APIs and micro-services architectures
  • Experience in working with CI/CD
  • Experience in working with SQL database like Postgres, Oracle
  • Preferably with hands-on experience with Hadoop big data tools (Hive, Impala, Spark)
  • Experience with data pipeline and workflow management tools: NIFI, Airflow.
  • Comfortable in developing shell scripts for automation.
  • Good troubleshooting and debugging skills.
  • Proficient in standard software development, such as version control, testing, and deployment
  • Demonstrated basic knowledge of statistical analytical techniques, coding, and data engineering
  • Ability to quickly learn and implement new technologies
  • Ability to Solve complex problems with multi-layered data sets
  • Ability to innovate and determine new approaches & technologies to solve business problems and generate business insights & recommendations.
  • Ability to multi-task and strong attention to detail
  • Flexibility to work as a member of a matrix based diverse and geographically distributed project teams
  • Good communication skills - both verbal and written - and strong relationship, collaboration skills, and organizational skills


Nice-to-Have Skills Description:

  • Experience with performance Tuning of Database Schemas, Databases, SQL, ETL Jobs, and related scripts
  • Experience in working with Cloud APIs (e.g., Azure, AWS)
  • Experience participating in complex engineering projects in an Agile setting e.g. Scrum

  • Big data developer

    1 day ago

    India IITMatrix Full time

    Work location : Singapore Mandate skills: Java, microservices, big data, spring boot TEAM Architecture (Big Data) Experience with application development in Big Data Ecosystem (such as Cloudera, Elastic Search, Spark, Presto, Hive, Impala, Kibana, Logstash, Hbase, Kafka), Expertise in building applications using Scala, Java, Python, Shell...

  • Big Data Developer

    9 hours ago

    India Impetus Full time

    Lead Software Engineer Locations- Indore, Noida, Hyderabad Job Description We are looking for a Lead Big Data Developer to design and oversee the implementation of large-scale data solutions, lead a talented team, and ensure the efficient processing and analysis of massive datasets. Roles & Responsibilities Key Responsibilities ...

  • Big data developer

    4 weeks ago

    India Manuh Technologies Full time

    Big Data Engineer with 5+ years of experience on technologies Cloudera, Hadoop, sqoop, Spark, Py Spark, Impala, SQL Previous experince into big data projects using hadoop and spark. Atleast one project experience into Cloudera hadoop with Impala and Hue Strong Programming experience in Python and Py Spark Hands on experience in SQL using Impala ...

  • Big Data Developer

    4 days ago

    India Luxoft Full time

    Project Description: As a Senior Engineer in the Data Engineering & Analytics team, you will develop data & analytics solutions that sit atop vast datasets gathered by retail stores, restaurants, banks, and other consumer-focused companies. The challenge will be to create high-performance algorithms, cutting-edge analytical techniques including machine...

  • Big Data Developer

    4 weeks ago

    India Manuh Technologies Full time

    Big Data Engineer with 5+ years of experience on technologies Cloudera, Hadoop, sqoop, Spark, PySpark, Impala, SQLPrevious experince into big data projects using hadoop and spark.Atleast one project experience into Cloudera hadoop with Impala and HueStrong Programming experience in Python and PySparkHands on experience in SQL using ImpalaExperience in...

  • Big Data Developer

    4 weeks ago

    India Manuh Technologies Full time

    Big Data Engineer with 5+ years of experience on technologies Cloudera, Hadoop, sqoop, Spark, PySpark, Impala, SQL Previous experince into big data projects using hadoop and spark. Atleast one project experience into Cloudera hadoop with Impala and Hue Strong Programming experience in Python and PySpark Hands on experience in SQL using Impala Experience in...

  • Big Data Developer

    1 month ago

    india Manuh Technologies Full time

    Big Data Engineer with 5+ years of experience on technologies Cloudera, Hadoop, sqoop, Spark, PySpark, Impala, SQLPrevious experince into big data projects using hadoop and spark.Atleast one project experience into Cloudera hadoop with Impala and HueStrong Programming experience in Python and PySparkHands on experience in SQL using ImpalaExperience in...

  • Big Data Architect and Developer

    2 weeks ago

    India Atyeti Inc Full time

    About Atyeti IncWe are a leading tech company that thrives on innovation. Our mission is to push the boundaries of what is possible in big data technology.Job Title: Big Data Architect and DeveloperThe successful candidate will work closely with our team to design and develop scalable big data solutions using Spark and Hadoop.Salary:$120,000 - $180,000 per...

  • Big Data

    3 months ago

    india Myananta Technologies Full time

    At Myananta Technologies, we’re looking for a Big Data (Hadoop). The ideal candidate is excited by experimentation and looking for a new challenge that stretches their talents. Job descriptionFull-Time Big Data (Hadoop) DeveloperMyananta Technologies | Gurugram is hiring for “Big Data (Hadoop) Developer” Experience: 3 - 5 Years.Salary: 5.0 - 12.0...

  • Big Data Architect

    7 months ago

    India Princeton IT Services Full time

    **Job Title**: Big Data Architect **Location**:Kakinada (work from office) or Bangalore (work from office) **Shift**: UK shift preferred **Job Overview**: We are seeking a highly skilled and motivated Big Data Architect to join our team. This individual will play a crucial role in designing and implementing scalable and efficient big data solutions,...

  • Big Data Developer

    11 hours ago

    India Tata Consultancy Services Full time

    Desired Competencies (Technical/Behavioral Competency) Must-Have •5+ years of active development experience with Java, Scala, Spark, Hive, Kafka, NoSql Databases •Having core fundamentals experience of Apache Hadoop components •Understanding best practices for Big Data (in Hadoop), data processing •Strong experience and debugging skills in Java,...

  • Big Data Developer

    9 hours ago

    India Tata Consultancy Services Full time

    Desired Competencies (Technical/Behavioral Competency) Must-Have •5+ years of active development experience with Java, Scala, Spark, Hive, Kafka, NoSql Databases •Having core fundamentals experience of Apache Hadoop components •Understanding best practices for Big Data (in Hadoop), data processing •Strong experience and debugging skills in Java,...

  • Big Data Developer

    3 weeks ago

    India Quess IT Staffing Full time

    Experience as a Big Data Developer and experienced in Data Ingestion with strong track record of designing and implementing data solutions. • Experience in programming language as Python or Java, with experience in building data pipelines and workflows. • Experience with cloud data warehousing technologies, such as Snowflake and Redshift. •...

  • Big Data Engineer

    3 weeks ago

    India Pert Telecom Solutions Full time

    Job Description : We are seeking a skilled Big Data Engineer with deep expertise in PySpark, Python, MySQL, Hadoop and Linux. The ideal candidate should have extensive experience deploying and managing these tools within on-premises environments (no public cloud). Proficiency in cluster management, performance optimization, and monitoring is essential...

  • Big Data Developer

    1 month ago

    india Impetus Full time

    F2F Hiring Drive on 16th Nov in Pune.Location: Impetus Technologies India Pvt. Ltd3rd Floor, We Work, Raheja WoodsKalyani Nagar, Pune – 411006 (Maharashtra)Exp: 3.5-6yrsExperience with design and coding across one or more platforms and languages (e.g. , Python/Pyspark/SQL, Scala) as appropriateHands-on expertise with application design, software...

  • Senior Big Data Architect

    2 weeks ago

    India Atyeti Inc Full time

    At Atyeti Inc, we are seeking a highly skilled Senior Big Data Architect to join our team. In this role, you will be responsible for analyzing and processing large datasets using big data technologies and programming languages such as Scala and Java.About the JobThe ideal candidate should have at least 6 years of experience in high-tech industry and/or IT...

  • Big Data Developer

    12 hours ago

    India Luxoft Full time

    Project Description: We've been engaged by a large UK Based Investment Bank in the Corporate, Commercial and Institutional Banking (CCIB) potfolio to work on the projects for managing analytics support for all the key initiatives of all segments/products . Responsibilities: Ensuring quality service output, submitted in a timely manner and...

  • Big Data Architect

    2 months ago

    india Affine Full time

    Primary Role:Resource and Project Timeline EstimationDefining project and program management processPrioritizing task within project: Task allocation across resources – capable of evaluating resources by their skills and improvement areas - prioritizes resource developmentContributes to org-level activities such as sales collaterals, designing proposals,...

  • Big Data Architect

    2 months ago

    india Affine Full time

    Primary Role : Resource and Project Timeline Estimation Defining project and program management process Prioritizing task within project: Task allocation across resources – capable of evaluating resources by their skills and improvement areas - prioritizes resource development Contributes to org-level activities such as sales collaterals, designing...

  • Senior Big Data Developer

    2 months ago

    india Synechron Full time

    Good-day, We have immediate opportunity for Senior Big Data Developer– 5 to 10 years Job Role : Senior Big Data Developer Job Location : Pune Experience : 5 to 10 years Notice Period : Immediate to 15 Days About Company: At Synechron, we believe in the power of digital to transform businesses for the better. Our global consulting firm combines...