Studios - Data Engineer - Hive/Spark

2 weeks ago

Bengaluru, Karnataka, India CAW Full time ₹ 20,00,000 - ₹ 25,00,000 per year

Job Description
We are looking for a savvy Data Engineer to join our growing team, responsible for expanding and optimizing our data and data pipeline architecture, as well as optimizing data flow. The ideal candidate is an experienced data pipeline builder and data wrangler who enjoys optimizing data systems and building them from the ground up. The right candidate will be excited by the prospect of optimizing or even re-designing our company's data architecture to support our next generation of products and data :

  • Create and maintain optimal data pipeline architecture.
  • Create and maintain events/streaming-based architecture/design.
  • Identify, design, and implement internal process improvements : automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
  • Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and AWS big data technologies.
  • Create data tools for analytics and data scientist team members that assist them in building

and optimizing our product into an innovative industry leader.

  • Work with data scientists to strive for greater functionality in our data systems.

Requirements

  • We are looking for a candidate with 2+ years of experience in a Data Engineer role.
  • Experience with big data tools : HDFS/S3 Spark/Flink, Hive, Hbase, Kafka/Kinesis, etc.
  • Experience with relational SQL and NoSQL databases, including Elasticsearch and Cassandra/Mongodb.
  • Experience with data pipeline and workflow management tools : Azkaban, Luigi, Airflow, etc.
  • Experience with AWS /GCP cloud services.
  • Experience with stream-processing systems : Spark-Streaming/Flink, etc.
  • Experience with object-oriented/object function scripting languages : Java, Scala, etc.
  • Experience building and optimizing big data pipelines, architectures, and data sets.
  • Strong analytic skills related to working with structured/unstructured datasets.
  • Build processes supporting data transformation, data structures, dimensional modeling,

metadata, dependency, schema registration/evolution, and workload management.

  • Working knowledge of message queuing, stream processing, and highly scalable big data data

stores.

  • Experience supporting and working with cross-functional teams in a dynamic environment.
  • Have a few weekend side projects up on GitHub.
  • Have contributed to an open-source project.
  • Have worked at a product company.
  • Have a working knowledge of a backend programming language.

)

  • Data Engineer- Spark

    6 days ago

    Bengaluru, Karnataka, India The It Mind Services Full time ₹ 4,00,000 - ₹ 12,00,000 per year

    Data Engineer - SPARKApache Spark (Scala), Hadoop, and Hive, performance tuning, debugging, and optimization , Functional Programming, OOP principles, SOLID design, and design patterns, RDBMS, NoSQL database (preferably PostgreSQL), build tools (Maven), version control systems (Git), and CI/CD pipelines (Jenkins preferred), unit and integration tests using...

  • Big Data Engineer – Spark

    2 weeks ago

    Bengaluru, Karnataka, India POWER IT SERVICES Full time ₹ 6,00,000 - ₹ 12,00,000 per year

    Big Data Engineer – Spark & AnalyticsLocation- ( Bangalore, Maharashtra)Job Description•Gather operational intel on business processes and policies from multiple sources•Prepare periodical and ad-hoc reports using operational data•Develop semantic core to align data with business processes•Support operations team's work streams for data processing,...

  • Data Engineer

    1 week ago

    Bengaluru, Karnataka, India NTT DATA Full time ₹ 6,00,000 - ₹ 18,00,000 per year

    As a Data Engineer, your main responsibilities will involve: The project is a migration of a SAS code to DBT code, so it's important that the candidate has experience in this type of projects. Analyze data models and derive logical conclusions. Processes modeling. Hands on development and monitoring of the Azure cloud Platform and various associated...

  • Data Engineer

    2 weeks ago

    Bengaluru, Karnataka, India Tata Consultancy Services Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Data Engineer - Senior Engineer/ManagerExperience Range: yearsLocation: Bengaluru, IndiaPrimary Skills Required:Position 1(Required Exp Range years): Java, Scala, Spark, Python, Kafka, Airflow, SQL, Databricks, Snowflake, Hive, AWS, Oracle Exadata, SAP Data ServicesPosition 2(Required Exp Range years): Java, Scala, Spark, Python, Kafka, Airflow, SQL,...

  • spark scala

    3 weeks ago

    Bengaluru, Karnataka, India, Karnataka Tata Consultancy Services Full time

    Hi,Greetings from TCS Team!Role Spark Scala DeveloperRequired Technical Skill Set Spark Scala - Coding knowledge expectedDesired Experience Range 6-9 YearsLocation of Requirement TCS - Chennai, Bangalore, Kochi, Pune, BhubaneswarIf you’re interested in the role if the experience, location and skillset match your profile then please share your updated...

  • Big Data Engineers/Senior Engineers

    3 days ago

    Bengaluru, Karnataka, India Peoplelogic Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Role: Big Data Engineers/Senior Engineers About the Role:We are seeking highly skilled Big Data Engineers/Senior Engineers with a strong background inbuilding and supporting big data solutions. The ideal candidate will have expertise in big datatechnologies, data pipeline development, and performance optimization within a collaborativeand agile...

  • Big Data Engineer

    6 days ago

    Bengaluru, Karnataka, India Brillio Full time ₹ 6,00,000 - ₹ 18,00,000 per year

    Key skills - Hive, SQL, Python, Spark, Hadoop and kafka , apache airflowJob Title Big Data EngineerRole OverviewWe are seeking a highly skilled Big Data Engineer to join our team. The ideal candidate will have strong experience in building, maintaining, and optimizing large-scale data pipelines and distributed data processing systems. This role involves...

  • Big Data Engineer

    1 day ago

    Bengaluru, Karnataka, India Deqode Full time ₹ 12,00,000 - ₹ 24,00,000 per year

    Profile: Big Data Engineer (System Design)Experience:5+ yearsLocation:BangaloreWork Mode:HybridAbout The RoleWe're looking for an experiencedBig Data Engineerwith system design expertise to architect and build scalable data pipelines and optimize big data solutions.Key ResponsibilitiesDesign, develop, and maintain data pipelines and ETL processes using...

  • Data Engineer

    2 weeks ago

    Bengaluru, Karnataka, India Digiquad Solutions Full time ₹ 4,00,000 - ₹ 12,00,000 per year

    2+ yrs exp as Data Engineer with strong skills in Big Data (HDFS/S3, Spark/Flink, Hive, HBase, Kafka/Kinesis), SQL & NoSQL (Elasticsearch, Cassandra, MongoDB), Airflow/Luigi, AWS/GCP, Java/Scala, stream processing & data modeling.

  • Data Engineer

    2 weeks ago

    Bengaluru, Karnataka, India Applore Technologies Full time ₹ 6,00,000 - ₹ 18,00,000 per year

    We are looking for a forward-thinking Data & AI Engineer with 1– 6 years of experience in data engineering and a passion for using modern AI tools to accelerate development workflows. The ideal candidate is proficient in Python, SQL, PySpark, and has experience working in on-premise big data environments (e.g., Spark, Hadoop, Hive, HDFS). This role is...