Big Data Developer

5 days ago

Bengaluru, India Luxoft Full time

Project Description:

As a Senior Engineer in the Data Engineering & Analytics team, you will develop data & analytics solutions that sit atop vast datasets gathered by retail stores, restaurants, banks, and other consumer-focused companies. The challenge will be to create high-performance algorithms, cutting-edge analytical techniques including machine learning and artificial intelligence, and intuitive workflows that allow our users to derive insights from big data that in turn drive their businesses. You will have the opportunity to create high-performance analytic solutions based on data sets measured in the billions of transactions and front-end visualizations to unleash the value of big data. You will have the opportunity to develop data-driven innovative analytical solutions and identify opportunities to support business and client needs in a quantitative manner and facilitate informed recommendations/decisions through activities like building ML models, automated data pipelines, designing data architecture/schema, performing jobs in big data cluster by using different execution engines and program languages such as Hive/Impala, Python, Java, Kafka, Spark, R, etc.


Responsibilities:

  • Hands-on developer who writes good quality, secure code that is modular, functional, and testable.
  • Drive the evolution of Data & Services products/platforms with an impact-focused on data science and engineering.
  • Design and implement scalable data architecture and data pipelines
  • Solving complex problems with multi-layered data sets, as well as optimizing existing machine learning libraries and frameworks.
  • Provide support for deployed data applications and analytical models by being a trusted advisor to Data Scientists and other data consumers by identifying data problems and guiding issue resolution with partner Data Engineers and source data providers.
  • Ensure proper data governance policies are followed by implementing or validating Data Lineage, Quality checks, classification, etc.
  • Discover, ingest, and incorporate new sources of real-time, streaming, batch, and API-based data into our platform to enhance the insights we get from running tests and expand the ways and properties on which we can test Experiment with new tools to streamline the development, testing, deployment, and running of our data pipelines.
  • Participate in the development of data and analytic infrastructure for product development
  • Continuously innovate and determine new approaches, tools, techniques & technologies to solve business problems and generate business insights & recommendations
  • Partner with roles across the organization including consultants, engineering, and sales to determine the highest priority problems to solve
  • Evaluate trade-offs between many possible analytics solutions to a problem, taking into account usability, technical feasibility, timelines, and differing stakeholder opinions to make a decision
  • Break large solutions into smaller, releasable milestones to collect data and feedback from product managers, clients, and other stakeholders
  • • Evangelize releases to users, incorporating feedback, and tracking usage to inform future development
  • Work with small, cross-functional teams to define the vision, establish team culture and processes
  • Consistently focus on key drivers of organization value and prioritize operational activities accordingly
  • Escalate technical errors or bugs detected in project work
  • Maintain awareness of relevant technical and product trends through self-learning/study, training classes, and job shadowing.
  • Support the building of scaled machine learning production systems by designing pipelines and engineering infrastructure.


Mandatory Skills Description:

  • Working proficiency in using Python/Scala, Spark (tuning jobs), SQL, Hadoop platforms to build Big Data products & platforms.
  • Good programming skills in Java and spring boot and Junit.
  • Knowledge in software development test approaches & frameworks
  • Familiarity with RESTful APIs and micro-services architectures
  • Experience in working with CI/CD
  • Experience in working with SQL database like Postgres, Oracle
  • Preferably with hands-on experience with Hadoop big data tools (Hive, Impala, Spark)
  • Experience with data pipeline and workflow management tools: NIFI, Airflow.
  • Comfortable in developing shell scripts for automation.
  • Good troubleshooting and debugging skills.
  • Proficient in standard software development, such as version control, testing, and deployment
  • Demonstrated basic knowledge of statistical analytical techniques, coding, and data engineering
  • Ability to quickly learn and implement new technologies
  • Ability to Solve complex problems with multi-layered data sets
  • Ability to innovate and determine new approaches & technologies to solve business problems and generate business insights & recommendations.
  • Ability to multi-task and strong attention to detail
  • Flexibility to work as a member of a matrix based diverse and geographically distributed project teams
  • Good communication skills - both verbal and written - and strong relationship, collaboration skills, and organizational skills


Nice-to-Have Skills Description:

  • Experience with performance Tuning of Database Schemas, Databases, SQL, ETL Jobs, and related scripts
  • Experience in working with Cloud APIs (e.g., Azure, AWS)
  • Experience participating in complex engineering projects in an Agile setting e.g. Scrum

  • Big Data Developer

    8 months ago

    Bengaluru, India Unison Consulting Pte Ltd Full time

    Job Description:Unison Consulting Pte Ltd is seeking an experienced Big Data Developer to join our team. As a Big Data Developer, you will be responsible for designing, developing, and maintaining big data solutions to enable data-driven decision making and analytics capabilities for our clients. You will work with large volumes of structured and...

  • Big Data Developer

    4 weeks ago

    Bengaluru, India Advent Global Solutions Full time

    Position : Big Data Developer/Big Data Senior DeveloperLocation : Chennai, Bangalore, Pune, Gurgaon, and MumbaiExperience : 5 to 14 yearsRelevant : should be 4+ yrs for Junior and 7+ yrs for SeniorPrimary skills : Spark, SQL,HiveDevelop and maintain architectures such as databases and large-scale processing systems using Python, Scala, SQL, and Unix...

  • Big Data Developer

    4 weeks ago

    Bengaluru, India Tata Consultancy Services Full time

    TCS has always been in the spotlight for being adept in "the next big technologies" What we can offer you is a space to explore varied technologies and quench your techie soul.Role: Big Data DeveloperLocation: Bangalore/Chennai/Hyderabad/PuneExperience:5-10 Years- Required Technical skills: Bigdata, Hadoop Technologies- 4+ years of experience in Big Data,...

  • Big data developer

    4 weeks ago

    Bengaluru, India Tata Consultancy Services Full time

    TCS has always been in the spotlight for being adept in "the next big technologies" What we can offer you is a space to explore varied technologies and quench your techie soul.Role: Big Data DeveloperLocation: Bangalore/Chennai/Hyderabad/PuneExperience:5-10 YearsRequired Technical skills: Bigdata, Hadoop Technologies4+ years of experience in Big Data,...

  • Big Data Developer

    3 weeks ago

    Bengaluru, India LTIMindtree Full time

    Please fill this form for quick responses ;-If it is not able to access please past it on Google and try to fill it...Link:- Big Data Developer/ Big Data EngineerSkills:- Pyspark, Sql, Hive,Exp:- 5 + YearsLoc:- Bengaluru, NCR Noida, Gurugram, DelhiMode;- HybridRegards,Dhamodhar. GTA

  • Big Data Developer

    4 weeks ago

    Bengaluru, India Tata Consultancy Services Full time

    TCS has always been in the spotlight for being adept in "the next big technologies" What we can offer you is a space to explore varied technologies and quench your techie soul.Role: Big Data DeveloperLocation: Bangalore/Chennai/Hyderabad/PuneExperience:5-10 YearsRequired Technical skills: Bigdata, Hadoop Technologies4+ years of experience in Big Data,...

  • Big data developer

    3 weeks ago

    Bengaluru, India LTIMindtree Full time

    Please fill this form for quick responses ;-If it is not able to access please past it on Google and try to fill it...Link:- Lx Fw Yq9Role:- Big Data Developer/ Big Data EngineerSkills:- Pyspark, Sql, Hive,Exp:- 5 + YearsLoc:- Bengaluru, NCR Noida, Gurugram, DelhiMode;- HybridRegards,Dhamodhar. GTA

  • Big Data Developer

    3 weeks ago

    Greater Bengaluru Area, India Pixomania Full time

    Job Title: Big Data Developer (Scala + Spark)Location: RemoteExperience: 5+ yearsJob Type: Contractual RemoteAbout Us:We are a forward-thinking technology company committed to delivering innovative solutions in Big Data analytics. Join our talented team as a Big Data Developer with expertise in Scala and Apache Spark to drive impactful projects and...

  • Big Data Developer

    3 weeks ago

    Greater Bengaluru Area, India Pixomania Full time

    Job Title: Big Data Developer (Scala + Spark)Location: RemoteExperience: 5+ yearsJob Type: Contractual RemoteAbout Us:We are a forward-thinking technology company committed to delivering innovative solutions in Big Data analytics. Join our talented team as a Big Data Developer with expertise in Scala and Apache Spark to drive impactful projects and...

  • Big Data Developer

    3 months ago

    Bengaluru, India Tata Consultancy Services Full time

    Greetings from TCS!TCS is hiring forBig Data DeveloperDesired Experience:5+ YearsJob Location:Chennai/ Bangalore / Pune / MumbaiRequire skillset:Big Data, Hadoop, Spark, Scala, HiveJob Description:-Must Have Technical Qualifications:SparkHadoopHiveScalaNice to Have Qualifications:Data Engineer CertificationExperience with Apache Beam.Technical...

  • Big Data Developer

    4 weeks ago

    Bengaluru, India Tata Consultancy Services Full time

    TCS has always been in the spotlight for being adept in "the next big technologies" What we can offer you is a space to explore varied technologies and quench your techie soul.Role: Big Data DeveloperLocation: Bangalore/Chennai/Hyderabad/PuneExperience:5-10 YearsRequired Technical skills: Bigdata, Hadoop Technologies4+ years of experience in Big Data,...

  • Big Data Developer

    4 months ago

    Bengaluru, India IBCScorp Full time

    Hi,We are hiring for Bigdata Developer for one of our Prestigious Client @ Bangalore Location. Job Location: BangalorePosition: Big Data DeveloperExperience: 5+yrsTotal YOE: 5+ Years Technical Experience 2+ Years of experience with Big Query SQL. 4+ years of experience with Scala and SPARK. Very strong in SQL 4+ yrs Experience with GCP Cloud. Candidate...

  • BIG DATA Developer

    1 month ago

    Bengaluru, India IBCScorp Full time

    Please see below the JD for Bigdata developer:Candidate should be available to interview in person in BangaloreWorking within a cross functional, agile development team, this Big Data Developer will be responsible for hands on development and technical delivery of innovative components used in data driven solutions.The candidate will have a sound working...

  • Big Data Developer

    3 weeks ago

    Bengaluru, India LTIMindtree Full time

    Please fill this form for quick responses ;- If it is not able to access please past it on Google and try to fill it... Link:- Role:- Big Data Developer/ Big Data Engineer Skills:- Pyspark, Sql, Hive, Exp:- 5 + Years Loc:- Bengaluru, NCR Noida, Gurugram, Delhi Mode;- Hybrid Regards, Dhamodhar. G TA

  • Big Data Developer

    4 weeks ago

    Bengaluru, India Luxoft Full time

    Project Description:The Cirrus project is developing cloud-based compliance state-of-the-art archive products to archive and retain real-time communications data in line with internal and external regulatory requirements. The product is developed in-house in an agile development setup, based on business requirements from multiple stakeholder parties. By...

  • Senior Big Data Developer

    2 months ago

    Bengaluru, India IntraEdge Full time

    We are looking for a highly skilled Senior Big Data Developer to join our engineering team. The successful candidate will be responsible for designing, developing, and maintaining large-scale data processing systems. You will work closely with cross-functional teams to architect scalable data pipelines and provide valuable insights through advanced data...

  • Senior Big Data Developer

    2 months ago

    Bengaluru, India IntraEdge Full time

    We are looking for a highly skilled Senior Big Data Developer to join our engineering team. The successful candidate will be responsible for designing, developing, and maintaining large-scale data processing systems. You will work closely with cross-functional teams to architect scalable data pipelines and provide valuable insights through advanced data...

  • Senior Big Data Developer

    2 months ago

    Bengaluru, India IntraEdge Full time

    We are looking for a highly skilled Senior Big Data Developer to join our engineering team. The successful candidate will be responsible for designing, developing, and maintaining large-scale data processing systems. You will work closely with cross-functional teams to architect scalable data pipelines and provide valuable insights through advanced data...

  • Big Data Developer

    3 months ago

    Bengaluru, India Tata Consultancy Services Full time

    Greetings from TCS!TCS is hiring for Big Data DeveloperDesired Experience: 5+ YearsJob Location: Chennai/ Bangalore / Pune / MumbaiRequire skillset: Big Data, Hadoop, Spark, Scala, HiveJob Description:-Must Have Technical Qualifications:SparkHadoopHiveScalaNice to Have Qualifications: Data Engineer CertificationExperience with Apache Beam.Technical...

  • Big Data Developer

    3 months ago

    Bengaluru, India Tata Consultancy Services Full time

    Greetings from TCS! TCS is hiring for Big Data Developer Desired Experience: 5+ Years Job Location: Chennai/ Bangalore / Pune / Mumbai Require skillset: Big Data, Hadoop, Spark, Scala, Hive Job Description:- Must Have Technical Qualifications: Spark Hadoop Hive Scala Nice to Have Qualifications: Data Engineer Certification Experience with Apache...