Big Data Systems Engineer

11 hours ago

Hubli, Karnataka, India beBeeDataEngineering Full time ₹ 20,00,000 - ₹ 30,00,000

Job Title:

Data Engineering Specialist

About the Role:

We are seeking a highly skilled Data Engineering Specialist to join our team. The ideal candidate will have a strong background in data engineering and experience with designing, building, and maintaining large-scale data systems.

Key Responsibilities:

  • Create and maintain events/streaming based architecture/design
  • Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability
  • Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and AWS 'big data' technologies.
  • Create data tools for analytics and data scientist team members that assist them in building and optimizing our product into an innovative industry leader.

Mandatory Qualifications:

  • Proficiency in either Scala or Python, expertise in Spark, and experience with performance tuning and optimization. DBT is also mandatory.
  • Experience with big data tools: HDFS/S3, Spark/Flink,Hive,Hbase, Kafka/Kinesis, etc.
  • Experience with relational SQL and NoSQL databases, including Elasticsearch and Cassandra/Mongodb.
  • Experience with data pipeline and workflow management tools: Azkaban, Luigi, Airflow, etc.
  • Experience with AWS /GCP cloud services
  • Experience with stream-processing systems: Spark-Streaming/Flink etc.
  • Experience with object-oriented/object function scripting languages: Java, Scala, etc.

Benefits:

  • Strong analytic skills related to working with structured/unstructured datasets.
  • Build processes supporting data transformation, data structures, dimensional modeling, metadata, dependency, schema registration/evolution and workload management.
  • Working knowledge of message queuing, stream processing, and highly scalable 'big data' data stores.

Other Requirements:

  • Experience supporting and working with cross-functional teams in a dynamic environment
  • Have a few weekend side projects up on GitHub
  • Have contributed to an open-source project
  • Have worked at a product company
  • Have a working knowledge of a backend programming language

  • Big Data Engineering Expertise

    1 week ago

    Hubli, Karnataka, India beBeeDataEngineering Full time US$ 1,00,000 - US$ 1,50,000

    Job Title: Big Data Engineering ExpertiseAbout the Role:As a seasoned big data engineer, you will be responsible for designing, building, and maintaining high-performance data pipelines that process vast amounts of data.Key Responsibilities:

  • Senior Big Data Engineer

    18 hours ago

    Hubli, Karnataka, India beBeeData Full time US$ 1,00,000 - US$ 2,00,000

    Big Data ExpertOur client is seeking an experienced Big Data expert to join their team. The ideal candidate will have extensive knowledge of Big Data technologies and Google Cloud Platform services.Design, develop and optimize data pipelines using GCP BigQuery Dataflow and Apache Airflow to support large-scale data analytics.Utilize the Big Data Hadoop...

  • Lead Big Data Engineer

    2 days ago

    Hubli, Karnataka, India beBeeDataEngineering Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Data Engineering RoleJob DescriptionGCP Data Engineers are responsible for designing, implementing, and managing data pipelines on Google Cloud Platform (GCP) to extract valuable insights from large datasets.Key ResponsibilitiesData Pipeline Design and Development: Design and develop scalable data pipelines using GCP services such as Cloud Dataflow, Cloud...

  • Big Data Specialist

    18 hours ago

    Hubli, Karnataka, India beBeeData Full time ₹ 18,00,000 - ₹ 25,00,000

    Big Data Engineer - Job OpportunityWe are seeking a highly skilled professional with expertise in Big Data technologies and AWS Cloud Services. The ideal candidate will bring strong hands-on experience in designing, architecting, and implementing scalable data engineering solutions while driving innovation within the team.Design, develop, and optimize Big...

  • Senior Big Data Architect

    3 days ago

    Hubli, Karnataka, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,20,00,000

    Big Data Systems DeveloperWe are seeking a skilled Data Engineer II to join our team. This role is crucial in designing and executing large-scale distributed data systems.The ideal candidate should have 2 to 4 years of experience in big data technologies, such as Apache Hadoop, and relational databases, including MS SQL Server, Oracle, MySQL, and...

  • Senior Big Data Architect

    1 day ago

    Hubli, Karnataka, India beBeeData Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Big Data Engineer PositionJob Summary:We are seeking a highly skilled Big Data Engineer to design and implement scalable data transformation pipelines using Snowflake and dbt, following industry-accepted best practices.Key Responsibilities:Data Pipeline Development: Design and implement robust ELT pipelines using dbt on Snowflake, ensuring scalability and...

  • Big Data Specialist

    3 days ago

    Hubli, Karnataka, India beBeeDataScientist Full time ₹ 60,00,000 - ₹ 1,20,00,000

    Job OverviewThe role involves working with large-scale data sets utilizing Big Data tools and technologies. We are looking for a skilled professional to join our team as a Data Scientist.Key Responsibilities:Develop, tweak queries, and work on performance enhancement using Scala, Spark, Hive, and Hadoop Big Data Eco System Tools.Apply SQL experience to...

  • Big Data Solutions Developer

    1 day ago

    Hubli, Karnataka, India beBeeDataEngineer Full time ₹ 1,80,00,000 - ₹ 2,50,00,000

    Data Engineer PositionJob Summary:We are seeking an experienced data engineer to design and implement large-scale data processing systems using Azure Databricks.Main Responsibilities:Develop and maintain high-quality data pipelines utilizing Azure Databricks, Python, and SQL.Collaborate with cross-functional teams to identify business requirements and create...

  • Big Data Solutions Architect

    2 weeks ago

    Hubli, Karnataka, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 25,00,000

    About the RoleA highly skilled Big Data Engineer is required to lead a team of big data engineers. This individual will work closely with clients and team members to understand their requirements and develop architectures that meet their needs.Key Responsibilities:Support the Data Engineering team in setting up the Data Lake on Cloud and implementing...

  • Big Data Specialist

    1 week ago

    Hubli, Karnataka, India beBeeData Full time ₹ 15,00,000 - ₹ 25,00,000

    Seeking a skilled professional to fill a Big Data positionJob Overview:We are looking for an experienced individual with expertise in processing and analyzing large datasets.Key Responsibilities:Data Ingestion: Design and implement efficient data ingestion processes from diverse sources onto our Big Data platformData Processing: Utilize PySpark, Hive, and...