Big Data Architect

2 weeks ago

Bengaluru, Karnataka, India Pylon Management Consulting Full time

Job Description We are seeking a highly skilled Senior Big Data Engineer with expertise in designing, optimizing, and managing large-scale Big Data systems. The ideal candidate will possess a deep understanding of Big Data platforms, cloud technologies, and DevOps practices, along with strong leadership and advisory skills to guide customers on strategic initiatives. This role requires hands-on experience with industry-standard tools, programming languages, and Big Data frameworks. Key Responsibilities : - Big Data System Design and Optimization Architect, develop, and optimize Big Data systems on Cloudera platforms (cloud or on-premise). - Design Hadoop clusters, including capacity planning, performance tuning, and data ingestion strategies. - Evaluate and implement scalable data pipeline solutions for batch and real-time processing. - Provide architectural guidance on data modeling, storage, and partitioning for optimal performance. - Data Migration and Validation Develop comprehensive data and metadata migration plans, ensuring high accuracy and efficiency. - Design and implement data validation and reconciliation methods. - Identify migration dependencies, risks, and success criteria, and devise mitigation strategies. - Cloud and DevOps Enablement Leverage expertise in cloud platforms (AWS, Azure, or Google Cloud) for scalable Big Data solutions. - Implement DevOps practices, including Git, CI/CD pipelines, and Infrastructure as Code (IaC). - Automate workflows and optimize resource utilization to reduce operational costs. - Big Data Toolset Expertise Proficiently utilize Apache Spark, Hive, Impala, and Kafka for data processing and streaming. - Hands-on experience with HBase, Apache Storm, Knox, Ozone, Iceberg, and Ambari. - Develop and manage data pipelines using Python, Scala, and relevant scripting languages. - Monitor, debug, and optimize distributed systems to ensure reliability and scalability. - Enterprise Security and Compliance Integrate enterprise security solutions such as LDAP, Kerberos, and role-based access controls. - Implement data encryption, masking, and secure access protocols to meet compliance standards. - Ensure adherence to security frameworks like GDPR, HIPAA, or other industry regulations. - Collaboration and Mentoring Collaborate with cross-functional teams to align on project goals and deliverables. - Mentor and upskill team members, fostering a culture of continuous learning. - Conduct workshops and training sessions to share knowledge and best practices. - Support and Troubleshooting Provide ongoing support for "Run The Engine" activities, ensuring system reliability. - Troubleshoot and resolve performance bottlenecks, failures, and scalability challenges. - Conduct root cause analysis for production incidents and implement preventive measures. Required Skills and Qualifications : Core Technical Expertise Big Data Frameworks : - Apache Spark, Hive, Impala, Kafka, HBase. Programming Languages : - Python, Scala, Java, Bash, or Perl. Cloud Platforms : - AWS, Azure, Google Cloud Platform (GCP). Data Management : - Expertise in Hadoop Distributed File System (HDFS), Apache Iceberg, and NoSQL databases. Security Solutions : - LDAP, Kerberos, encryption techniques, and access control mechanisms. - DevOps and Automation Skills Proficiency with Git, CI/CD tools (Jenkins, GitLab, CircleCI), and Infrastructure as Code (Terraform, CloudFormation). - Automation experience using Ansible, Chef, Puppet, or similar tools. - Exposure to containerization and orchestration tools like Docker and Kubernetes. - Analytical and Problem-Solving Skills Strong ability to design efficient data models and optimize large-scale data pipelines. - Exceptional problem-solving skills to debug and resolve complex technical issues. - Proficiency in monitoring and logging tools such as Grafana, Prometheus, or Splunk. - Preferred Skills Knowledge of data lake architectures and advanced analytics solutions. - Experience with real-time data streaming frameworks like Flink or Beam. - Familiarity with machine learning workflows and tools (TensorFlow, PyTorch, or Spark MLlib). - Experience working with international clients in Europe or the US region (ref:hirist.tech)

  • Senior Big Data Solutions Architect

    4 days ago

    Bengaluru, Karnataka, India NTT DATA Global Delivery Services Limited Full time

    Job Description:We are seeking a Senior Big Data Solutions Architect to lead our big data initiatives. The successful candidate will design, build, and maintain scalable data pipelines using Databricks on AWS. This includes data integration, data processing, and data architecture. The ideal candidate will have strong experience with Databricks, including...

  • Big Data Architect

    5 days ago

    Bengaluru, Karnataka, India Blend Full time

    As a Big Data Architect at Blend, you'll play a critical role in designing and implementing data solutions that meet the needs of our clients.About the RoleWe're seeking a skilled Big Data Architect to join our team and deliver exceptional results for our clients. As a key member of our Data Engineering team, you'll be focused on designing, implementing, and...

  • Big Data Solutions Architect

    5 days ago

    Bengaluru, Karnataka, India Tata Consultancy Services Full time

    Job Summary:We are seeking a highly skilled Big Data Solutions Architect to lead the design and implementation of our big data solutions. You will work closely with cross-functional teams to develop innovative solutions that leverage big data technologies such as Hadoop, Spark, and NoSQL databases. The ideal candidate will have a strong background in big...

  • Big Data Solutions Architect

    1 day ago

    Bengaluru, Karnataka, India Emorphis Technologies Pvt. Ltd. Full time

    Job OverviewWe are seeking a Big Data Solutions Architect to lead the development of big data solutions at Emorphis Technologies Pvt. Ltd.Main Responsibilities:Design and implement big data architectures using AWS services.Develop expertise in data warehousing and ETL processes.Collaborate with cross-functional teams to identify business needs and develop...

  • Big Data Solutions Architect

    1 week ago

    Bengaluru, Karnataka, India Tata Consultancy Services Full time

    About UsTata Consultancy Services (TCS) is a leading IT services company that helps clients transform their businesses through technology. We are committed to delivering high-quality solutions that meet the needs of our clients.We are seeking an experienced Big Data Solutions Architect to join our team. As a key member of our organization, you will be...

  • AWS Big Data Solution Architect

    4 days ago

    Bengaluru, Karnataka, India Birlasoft Limited Full time

    Job Title: AWS Big Data Solution ArchitectCompany Overview:Birlasoft Limited is a leading global IT services company that helps clients transform their businesses through digital innovations. We are looking for an experienced AWS Big Data Solution Architect to join our team.Job Description:We are seeking a highly skilled and motivated AWS Big Data Solution...

  • Big Data Architect

    6 days ago

    Bengaluru, Karnataka, India IntraEdge Full time

    Job Description:IntraEdge is looking for a Big Data Architect - Java Expert to join our team. This individual will be responsible for designing and implementing large-scale big data architectures, working closely with our development team to ensure that our software applications are scalable, efficient, and reliable. The ideal candidate will have a strong...

  • Azure Big Data Architect

    15 hours ago

    Bengaluru, Karnataka, India Tiger Analytics Full time

    About Tiger Analytics:We are a global analytics consulting firm driving the AI revolution. Our expertise lies in building innovative solutions that energize and inspire.With delivery centers across the globe, we are headquartered in the Silicon Valley.About the Role:We are seeking a seasoned Data Architect / Sr. Data and Principal Data Architects to be based...

  • Big Data Solutions Architect

    7 days ago

    Bengaluru, Karnataka, India Impetus Full time

    Impetus is a cutting-edge organization that specializes in providing innovative data solutions to its clients. We are currently seeking an experienced Data Engineering Specialist to join our team and contribute to the design and implementation of large-scale data solutions.About the Role:This is a critical role within our organization, and the successful...

  • Big Data Solution Architect

    1 day ago

    Bengaluru, Karnataka, India Ascendion Full time

    We are looking for a talented Big Data Solution Architect to help us design and implement scalable MongoDB solutions.In this role, you will be responsible for configuring and optimizing MongoDB's performance, as well as performing advanced engineering tasks to ensure efficient data storage and retrieval.The ideal candidate will have strong knowledge of big...

  • Big Data Solutions Architect

    18 hours ago

    Bengaluru, Karnataka, India Deltaclass Technology Solutions Pvt. Ltd. Full time

    Deltaclass Technology Solutions Pvt. Ltd. is a technology-driven company that specializes in providing innovative solutions to its clients.We are seeking an experienced Big Data Solutions Architect to join our team.As a key member of our engineering team, you will be responsible for designing and developing scalable big data solutions using Apache Spark.Key...

  • Big Data Architect and Engineer

    6 days ago

    Bengaluru, Karnataka, India Andor Tech Full time

    About Andor TechWe are a leading provider of innovative big data solutions. Our team is dedicated to delivering exceptional results through expertise and passion.Job DescriptionWe are seeking an experienced Big Data Architect and Engineer to join our team. The ideal candidate will have 8+ years of experience in designing and developing scalable data...

  • AWS Big data Architect

    2 weeks ago

    Bengaluru, Karnataka, India Birlasoft Limited Full time

    Job DescriptionKey Responsibilities- Design and architect end to end solutions on AWS and create the HLD and LLD documents- Experience in real-time Data Ingestion and Processing- Hands-on experience in AWS S3, Kafka, Glue, PySpark, Python, Redshift, AWS Cloud best practices and optimization- Experience with integration of different data sources with Data...

  • Big Data Architect

    7 days ago

    Bengaluru, Karnataka, India Tata Consultancy Services Full time

    Why Join Us?TCS is a global leader in the technology industry, providing exciting career opportunities for talented individuals. As a Big Data Developer, you'll have the chance to work on challenging projects, develop new skills, and grow professionally. Our company culture emphasizes innovation, teamwork, and continuous learning.Key Responsibilities:-...

  • Big data developer

    7 days ago

    Bengaluru, Karnataka, India Impetus Full time

    Big Data DeveloperExperience- 3 to 12 yearsLocations- Indore, Noida, Hyderabad, Bangalore, GurgaonJob DescriptionWe are looking for a Big Data Developer to design and oversee the implementation of large-scale data solutions, lead a talented team, and ensure the efficient processing and analysis of massive datasets.Roles & ResponsibilitiesKey...

  • Big Data Developer

    4 weeks ago

    Bengaluru, Karnataka, India Risk Resources Full time

    Overview : The Bigdata Developer plays a vital role in today's data-driven organizations by enabling the extraction, transformation, and loading (ETL) of large volumes of data from diverse sources to generate actionable insights. This position leverages advanced technologies that support the storage, processing, and analysis of big data. Bigdata Developers...

  • Big Data Developer

    1 week ago

    Bengaluru, Karnataka, India Prakhar Software Solutions Private Limited Full time

    Objectives of this role : - Designing, developing, and implementing scalable and efficient data processing pipelines using Big Data technologies.- Collaborating with data architects, data scientists, and business analysts to understand and translate data requirements into technical solutions.- Developing and optimising ETL (Extract, Transform, Load)...

  • Data Architect

    4 weeks ago

    Bengaluru, Karnataka, India Tiger Analytics Full time

    Azure Big Data ArchitectChennai, Hyderabad, Bangalore.JOB DESCRIPTIONTiger Analytics is a global analytics consulting firm. With data and technology at the core of our solutions, we are solving some of the toughest problems out there. Our culture is modeled around expertise and mutual respect with a team first mindset.Working at Tiger, you'll be at the heart...

  • Data architect

    7 days ago

    Bengaluru, Karnataka, India Siemens Healthineers Full time

    As a Data Architect, you are required to:• Design & develop technical solutions which combine disparate information to create meaningful insights for business, using Big-data architectures• Build and analyze large, structured and unstructured databases based on scalable cloud infrastructures• Develop prototypes and proof of concepts using multiple...

  • Big Data Architect

    14 hours ago

    Bengaluru, Karnataka, India Tata Consultancy Services Full time

    Job DescriptionWe are seeking an experienced Python Developer to join our team. The ideal candidate will have a strong background in Python programming and experience with Big Data technologies such as SparkSQL and SparkStreaming.