1,116 Big Data jobs in India
Data Lake Architect
Hyderabad, Andhra Pradesh
Solix Technologies
Job Viewed
Job Description
**Careers at Solix**:
Careers at Solix offer the opportunity to join a passionate, innovative team driving enterprise data management. Collaborate on cutting-edge technologies, empower Fortune 500 clients, and grow professionally in a dynamic environment dedicated to excellence and digital transformation
**Join Our Winning Team**:
If you are looking for a fast paced environment filled with passionate collaborators and innovators, Solix is the place you are looking for.
We are a bunch of passionate creators and communicators who are set out to make enterprises data driven. Our teams work on cutting edge open source technologies to deliver groundbreaking big data management products to our customers across the world. Our customers are mostly category leaders and include many Fortune 500 companies. They, like us, believe being data driven is the only way to deliver awesome experiences to customers while driving value to shareholders.
**Data Lake Architect**Location: Hyderabad, India | Experince: 10-15+ Years**:
**Job Duties and Responsibilities**:
As a Data Lake Architect with 10-15+ years of experience in Java and Spring, you will be responsible for designing and implementing data lake architectures for large-scale, complex data environments using Java and Spring technologies. You will collaborate with cross-functional teams to develop and implement solutions that meet the business requirements for data storage, processing, and analysis. You will have a deep understanding of data architecture principles and best practices, as well as strong proficiency in Java and Spring, and be able to lead the development of scalable, efficient, and secure data lake environments.
**Key Responsibilities**:
- Design and develop Data Lake Architectures that support the storage, processing, and analysis of large-scale, complex data environments using Java and Spring technologies.
- Collaborate with cross-functional teams to gather and analyze requirements and develop data lake solutions that meet business needs.
- Develop and implement data governance policies and procedures to ensure the security, accuracy, and reliability of data within the data lake.
- Lead the development of data ingestion, transformation, and processing pipelines using Java and Spring technologies that enable data to be processed and analyzed efficiently.
- Work with infrastructure and operations teams to ensure the data lake is scalable, reliable, and efficient.
- Develop and implement monitoring and alerting systems to proactively identify and resolve issues within the data lake environment.
- Keep up to date with emerging technologies and trends in data architecture, Java and Spring development, and identify opportunities for innovation and improvement.
**Qualifications**:
- Bachelor's degree in Computer Science, Information Systems, or related field.
- 10-15+ years of experience in data architecture and design, with a focus on large-scale, complex data environments using Java and Spring technologies.
- Strong understanding of data lake architecture principles and best practices.
- Proficiency in Java and Spring development.
- Experience with cloud-based data lake platforms such as AWS, Azure, or Google Cloud Platform is a plus.
- Experience designing and implementing data governance policies and procedures.
- Experience leading the development of data ingestion, transformation, and processing pipelines using Java and Spring technologies.
- Strong understanding of data access and control policies and procedures.
- Experience working with infrastructure and operations teams to ensure the data lake is scalable, reliable, and efficient.
- Strong communication and collaboration skills.
- Ability to lead cross-functional teams and work in a fast-paced, dynamic environment.
Careers at Solix offer the opportunity to join a passionate, innovative team driving enterprise data management. Collaborate on cutting-edge technologies, empower Fortune 500 clients, and grow professionally in a dynamic environment dedicated to excellence and digital transformation
**Join Our Winning Team**:
If you are looking for a fast paced environment filled with passionate collaborators and innovators, Solix is the place you are looking for.
We are a bunch of passionate creators and communicators who are set out to make enterprises data driven. Our teams work on cutting edge open source technologies to deliver groundbreaking big data management products to our customers across the world. Our customers are mostly category leaders and include many Fortune 500 companies. They, like us, believe being data driven is the only way to deliver awesome experiences to customers while driving value to shareholders.
**Data Lake Architect**Location: Hyderabad, India | Experince: 10-15+ Years**:
**Job Duties and Responsibilities**:
As a Data Lake Architect with 10-15+ years of experience in Java and Spring, you will be responsible for designing and implementing data lake architectures for large-scale, complex data environments using Java and Spring technologies. You will collaborate with cross-functional teams to develop and implement solutions that meet the business requirements for data storage, processing, and analysis. You will have a deep understanding of data architecture principles and best practices, as well as strong proficiency in Java and Spring, and be able to lead the development of scalable, efficient, and secure data lake environments.
**Key Responsibilities**:
- Design and develop Data Lake Architectures that support the storage, processing, and analysis of large-scale, complex data environments using Java and Spring technologies.
- Collaborate with cross-functional teams to gather and analyze requirements and develop data lake solutions that meet business needs.
- Develop and implement data governance policies and procedures to ensure the security, accuracy, and reliability of data within the data lake.
- Lead the development of data ingestion, transformation, and processing pipelines using Java and Spring technologies that enable data to be processed and analyzed efficiently.
- Work with infrastructure and operations teams to ensure the data lake is scalable, reliable, and efficient.
- Develop and implement monitoring and alerting systems to proactively identify and resolve issues within the data lake environment.
- Keep up to date with emerging technologies and trends in data architecture, Java and Spring development, and identify opportunities for innovation and improvement.
**Qualifications**:
- Bachelor's degree in Computer Science, Information Systems, or related field.
- 10-15+ years of experience in data architecture and design, with a focus on large-scale, complex data environments using Java and Spring technologies.
- Strong understanding of data lake architecture principles and best practices.
- Proficiency in Java and Spring development.
- Experience with cloud-based data lake platforms such as AWS, Azure, or Google Cloud Platform is a plus.
- Experience designing and implementing data governance policies and procedures.
- Experience leading the development of data ingestion, transformation, and processing pipelines using Java and Spring technologies.
- Strong understanding of data access and control policies and procedures.
- Experience working with infrastructure and operations teams to ensure the data lake is scalable, reliable, and efficient.
- Strong communication and collaboration skills.
- Ability to lead cross-functional teams and work in a fast-paced, dynamic environment.
This advertiser has chosen not to accept applicants from your region.
Job No Longer Available
This position is no longer listed on WhatJobs. The employer may be reviewing applications, filled the role, or has removed the listing.
However, we have similar jobs available for you below.
Big Data
Bengaluru, Karnataka
Manasio Pvt Ltd
Posted today
Job Viewed
Job Description
**Relevant Experience: 4+Yrs**
- **Joining details: Immediate/Within 15 Days**
- **Job Location: Bangalore**
- **Job Type: Contract ( Min 1 Yr.)**
- **Shift: Day Shift**
- **Salary: Best in the Industry**
Must Have
- Spark
- Scala
- Hive
- Hadoop
- SQL knowledge (intermediate)
Nice to have
- Sqoop
- Oozie
- Streaming
- Kafka
- Azure/Azure functions
- Databricks
- Airflow
**Job Type**: Contractual / Temporary
Contract length: 12 months
**Salary**: ₹8,088.09 - ₹41,464.62 per month
**Benefits**:
- Flexible schedule
- Work from home
Schedule:
- Flexible shift
- Morning shift
Ability to commute/relocate:
- Bengaluru, Bengaluru Urban - 560105, Karnataka: Reliably commute or planning to relocate before starting work (required)
Application Question(s):
- **Joining details: Immediate/Within 15 Days**
- **Job Location: Bangalore**
- **Job Type: Contract ( Min 1 Yr.)**
- **Shift: Day Shift**
- **Salary: Best in the Industry**
Must Have
- Spark
- Scala
- Hive
- Hadoop
- SQL knowledge (intermediate)
Nice to have
- Sqoop
- Oozie
- Streaming
- Kafka
- Azure/Azure functions
- Databricks
- Airflow
**Job Type**: Contractual / Temporary
Contract length: 12 months
**Salary**: ₹8,088.09 - ₹41,464.62 per month
**Benefits**:
- Flexible schedule
- Work from home
Schedule:
- Flexible shift
- Morning shift
Ability to commute/relocate:
- Bengaluru, Bengaluru Urban - 560105, Karnataka: Reliably commute or planning to relocate before starting work (required)
Application Question(s):
This advertiser has chosen not to accept applicants from your region.
0
Big Data
Chennai, Tamil Nadu
Virtusa
Posted today
Job Viewed
Job Description
**About Virtusa**
Teamwork, quality of life, professional and personal development: values that Virtusa is proud to embody. When you join us, you join a team of 27,000 people globally that cares about your growth — one that seeks to provide you with exciting projects, opportunities and work with state of the art technologies throughout your career with us.
Great minds, great potential: it all comes together at Virtusa. We value collaboration and the team environment of our company, and seek to provide great minds with a dynamic place to nurture new ideas and foster excellence.
Virtusa was founded on principles of equal opportunity for all, and so does not discriminate on the basis of race, religion, color, sex, gender identity, sexual orientation, age, non-disqualifying physical or mental disability, national origin, veteran status or any other basis covered by appropriate law. All employment is decided on the basis of qualifications, merit, and business need.
Teamwork, quality of life, professional and personal development: values that Virtusa is proud to embody. When you join us, you join a team of 27,000 people globally that cares about your growth — one that seeks to provide you with exciting projects, opportunities and work with state of the art technologies throughout your career with us.
Great minds, great potential: it all comes together at Virtusa. We value collaboration and the team environment of our company, and seek to provide great minds with a dynamic place to nurture new ideas and foster excellence.
Virtusa was founded on principles of equal opportunity for all, and so does not discriminate on the basis of race, religion, color, sex, gender identity, sexual orientation, age, non-disqualifying physical or mental disability, national origin, veteran status or any other basis covered by appropriate law. All employment is decided on the basis of qualifications, merit, and business need.
This advertiser has chosen not to accept applicants from your region.
1
Big Data Developer - Java, Big data, Spring
Hyderabad, Andhra Pradesh
Optum
Posted today
Job Viewed
Job Description
Primary Responsibilities:
- Analyzes and investigates
- Provides explanations and interpretations within area of expertise
- Participate in scrum process and deliver stories/features according to the schedule
- Collaborate with team, architects and product stakeholders to understand the scope and design of a deliverable
- Participate in product support activities as needed by the team.
- Understand product architecture, features being built and come up with product improvement ideas and POCs
- Comply with the terms and conditions of the employment contract, company policies and procedures, and any and all directives (such as, but not limited to, transfer and/or re-assignment to different work locations, change in teams and/or work shifts, policies in regards to flexibility of work benefits and/or work environment, alternative work arrangements, and other decisions that may arise due to the changing business environment). The Company may adopt, vary or rescind these policies and directives in its absolute discretion and without any limitation (implied or otherwise) on its ability to do so
Qualifications -
Required Qualifications:
- Undergraduate degree or equivalent experience
- Proven experience using Bigdata tech stack
- Sound knowledge on Java and Spring framework with good exposure to Spring Batch, Spring Data, Spring Web services, Python
- Proficient with Bigdata ecosystem (Sqoop, Spark, Hadoop, Hive, HBase)
- Proficient with Unix/Linux eco systems and shell scripting skills
- Proven Java, Kafka, Spark, Big Data, Azure ,analytical and problem solving skills
- Proven solid analytical and communication skills
This advertiser has chosen not to accept applicants from your region.
2
Big Data Developer - Java, Big data, Spring
Hyderabad, Andhra Pradesh
Optum
Posted today
Job Viewed
Job Description
Primary Responsibilities:
- Analyzes and investigates
- Provides explanations and interpretations within area of expertise
- Participate in scrum process and deliver stories/features according to the schedule
- Collaborate with team, architects and product stakeholders to understand the scope and design of a deliverable
- Participate in product support activities as needed by the team.
- Understand product architecture, features being built and come up with product improvement ideas and POCs
- Comply with the terms and conditions of the employment contract, company policies and procedures, and any and all directives (such as, but not limited to, transfer and/or re-assignment to different work locations, change in teams and/or work shifts, policies in regards to flexibility of work benefits and/or work environment, alternative work arrangements, and other decisions that may arise due to the changing business environment). The Company may adopt, vary or rescind these policies and directives in its absolute discretion and without any limitation (implied or otherwise) on its ability to do so
Qualifications -
Required Qualifications:
- Undergraduate degree or equivalent experience
- Proven experience using Bigdata tech stack
- Sound knowledge on Java and Spring framework with good exposure to Spring Batch, Spring Data, Spring Web services, Python
- Proficient with Bigdata ecosystem (Sqoop, Spark, Hadoop, Hive, HBase)
- Proficient with Unix/Linux eco systems and shell scripting skills
- Proven Java, Kafka, Spark, Big Data, Azure ,analytical and problem solving skills
- Proven solid analytical and communication skills
This advertiser has chosen not to accept applicants from your region.
3
Big Data Developer - Java, Big data, Spring
Hyderabad, Andhra Pradesh
Optum
Posted today
Job Viewed
Job Description
Primary Responsibilities:
Analyzes and investigates
Provides explanations and interpretations within area of expertise
Participate in scrum process and deliver stories/features according to the schedule
Collaborate with team, architects and product stakeholders to understand the scope and design of a deliverable
Participate in product support activities as needed by the team.
Understand product architecture, features being built and come up with product improvement ideas and POCs
Comply with the terms and conditions of the employment contract, company policies and procedures, and any and all directives (such as, but not limited to, transfer and/or re-assignment to different work locations, change in teams and/or work shifts, policies in regards to flexibility of work benefits and/or work environment, alternative work arrangements, and other decisions that may arise due to the changing business environment). The Company may adopt, vary or rescind these policies and directives in its absolute discretion and without any limitation (implied or otherwise) on its ability to do so
Qualifications -
Required Qualifications:
Undergraduate degree or equivalent experience
Proven experience using Bigdata tech stack
Sound knowledge on Java and Spring framework with good exposure to Spring Batch, Spring Data, Spring Web services, Python
Proficient with Bigdata ecosystem (Sqoop, Spark, Hadoop, Hive, HBase)
Proficient with Unix/Linux eco systems and shell scripting skills
Proven Java, Kafka, Spark, Big Data, Azure ,analytical and problem solving skills
Proven solid analytical and communication skills
Analyzes and investigates
Provides explanations and interpretations within area of expertise
Participate in scrum process and deliver stories/features according to the schedule
Collaborate with team, architects and product stakeholders to understand the scope and design of a deliverable
Participate in product support activities as needed by the team.
Understand product architecture, features being built and come up with product improvement ideas and POCs
Comply with the terms and conditions of the employment contract, company policies and procedures, and any and all directives (such as, but not limited to, transfer and/or re-assignment to different work locations, change in teams and/or work shifts, policies in regards to flexibility of work benefits and/or work environment, alternative work arrangements, and other decisions that may arise due to the changing business environment). The Company may adopt, vary or rescind these policies and directives in its absolute discretion and without any limitation (implied or otherwise) on its ability to do so
Qualifications -
Required Qualifications:
Undergraduate degree or equivalent experience
Proven experience using Bigdata tech stack
Sound knowledge on Java and Spring framework with good exposure to Spring Batch, Spring Data, Spring Web services, Python
Proficient with Bigdata ecosystem (Sqoop, Spark, Hadoop, Hive, HBase)
Proficient with Unix/Linux eco systems and shell scripting skills
Proven Java, Kafka, Spark, Big Data, Azure ,analytical and problem solving skills
Proven solid analytical and communication skills
This advertiser has chosen not to accept applicants from your region.
4
Big data developer - java, big data, spring
Optum
Posted today
Job Viewed
Job Description
Primary Responsibilities:Analyzes and investigatesProvides explanations and interpretations within area of expertiseParticipate in scrum process and deliver stories/features according to the scheduleCollaborate with team, architects and product stakeholders to understand the scope and design of a deliverableParticipate in product support activities as needed by the team.Understand product architecture, features being built and come up with product improvement ideas and POCsComply with the terms and conditions of the employment contract, company policies and procedures, and any and all directives (such as, but not limited to, transfer and/or re-assignment to different work locations, change in teams and/or work shifts, policies in regards to flexibility of work benefits and/or work environment, alternative work arrangements, and other decisions that may arise due to the changing business environment). The Company may adopt, vary or rescind these policies and directives in its absolute discretion and without any limitation (implied or otherwise) on its ability to do soQualifications - Required Qualifications:Undergraduate degree or equivalent experienceProven experience using Bigdata tech stackSound knowledge on Java and Spring framework with good exposure to Spring Batch, Spring Data, Spring Web services, PythonProficient with Bigdata ecosystem (Sqoop, Spark, Hadoop, Hive, HBase)Proficient with Unix/Linux eco systems and shell scripting skillsProven Java, Kafka, Spark, Big Data, Azure ,analytical and problem solving skillsProven solid analytical and communication skills
This advertiser has chosen not to accept applicants from your region.
5
Big Data Developer - Java, Big data, Spring
Hyderabad, Andhra Pradesh
Optum
Posted today
Job Viewed
Job Description
Primary Responsibilities:
- Analyzes and investigates
- Provides explanations and interpretations within area of expertise
- Participate in scrum process and deliver stories/features according to the schedule
- Collaborate with team, architects and product stakeholders to understand the scope and design of a deliverable
- Participate in product support activities as needed by the team.
- Understand product architecture, features being built and come up with product improvement ideas and POCs
- Comply with the terms and conditions of the employment contract, company policies and procedures, and any and all directives (such as, but not limited to, transfer and/or re-assignment to different work locations, change in teams and/or work shifts, policies in regards to flexibility of work benefits and/or work environment, alternative work arrangements, and other decisions that may arise due to the changing business environment). The Company may adopt, vary or rescind these policies and directives in its absolute discretion and without any limitation (implied or otherwise) on its ability to do so
Qualifications -
Required Qualifications:
- Undergraduate degree or equivalent experience
- Proven experience using Bigdata tech stack
- Sound knowledge on Java and Spring framework with good exposure to Spring Batch, Spring Data, Spring Web services, Python
- Proficient with Bigdata ecosystem (Sqoop, Spark, Hadoop, Hive, HBase)
- Proficient with Unix/Linux eco systems and shell scripting skills
- Proven Java, Kafka, Spark, Big Data, Azure ,analytical and problem solving skills
- Proven solid analytical and communication skills
This advertiser has chosen not to accept applicants from your region.
Be The First To Know
About the latest Big data Jobs in India !
6
Big Data Developer - Java, Big data, Spring
Hyderabad, Andhra Pradesh
Optum
Posted 6 days ago
Job Viewed
Job Description
Primary Responsibilities:
- Analyzes and investigates
- Provides explanations and interpretations within area of expertise
- Participate in scrum process and deliver stories/features according to the schedule
- Collaborate with team, architects and product stakeholders to understand the scope and design of a deliverable
- Participate in product support activities as needed by the team.
- Understand product architecture, features being built and come up with product improvement ideas and POCs
- Comply with the terms and conditions of the employment contract, company policies and procedures, and any and all directives (such as, but not limited to, transfer and/or re-assignment to different work locations, change in teams and/or work shifts, policies in regards to flexibility of work benefits and/or work environment, alternative work arrangements, and other decisions that may arise due to the changing business environment). The Company may adopt, vary or rescind these policies and directives in its absolute discretion and without any limitation (implied or otherwise) on its ability to do so
Qualifications -
Required Qualifications:
- Undergraduate degree or equivalent experience
- Proven experience using Bigdata tech stack
- Sound knowledge on Java and Spring framework with good exposure to Spring Batch, Spring Data, Spring Web services, Python
- Proficient with Bigdata ecosystem (Sqoop, Spark, Hadoop, Hive, HBase)
- Proficient with Unix/Linux eco systems and shell scripting skills
- Proven Java, Kafka, Spark, Big Data, Azure ,analytical and problem solving skills
- Proven solid analytical and communication skills
This advertiser has chosen not to accept applicants from your region.
7
Big Data Engineer
Pune, Maharashtra
Citigroup
Posted today
Job Viewed
Job Description
The Applications Development Programmer Analyst is an intermediate level position responsible for participation in the establishment and implementation of new or revised application systems and programs in coordination with the Technology team. The overall objective of this role is to contribute to applications systems analysis and programming activities.
**Responsibilities:**
+ Utilize knowledge of applications development procedures and concepts, and basic knowledge of other technical areas to identify and define necessary system enhancements
+ Identify and analyze issues, make recommendations, and implement solutions
+ Utilize knowledge of business processes, system processes, and industry standards to solve complex issues
+ Analyze information and make evaluative judgements to recommend solutions and improvements
+ Conduct testing and debugging, utilize script tools, and write basic code for design specifications
+ Assess applicability of similar experiences and evaluate options under circumstances not covered by procedures
+ Develop working knowledge of Citi's information systems, procedures, standards, client server application development, network operations, database administration, systems administration, data center operations, and PC-based applications
+ Appropriately assess risk when business decisions are made, demonstrating particular consideration for the firm's reputation and safeguarding Citigroup, its clients and assets, by driving compliance with applicable laws, rules and regulations, adhering to Policy, applying sound ethical judgment regarding personal behavior, conduct and business practices, and escalating, managing and reporting control issues with transparency.
**Qualifications:**
+ 3 to 5 years of relevant experience
+ Experience in programming/debugging used in business applications
+ Working knowledge of industry practice and standards
+ Comprehensive knowledge of specific business area for application development
+ Working knowledge of program languages
+ Consistently demonstrates clear and concise written and verbal communication
**Education:**
+ Bachelor's degree/University degree or equivalent experience
This job description provides a high-level review of the types of work performed. Other job-related duties may be assigned as required.
Additional Job Description
We are looking for a Big Data Engineer that will work on the collecting, storing, processing, and analyzing of huge sets of data. The primary focus will be on choosing optimal solutions to use for these purposes, then maintaining, implementing, and monitoring them. You will also be responsible for integrating them with the architecture used across the company.
Responsibilities
- Selecting and integrating any Big Data tools and frameworks required to provide requested capabilities
- Implementing data wrangling, scarping, cleaning using both Java or Python
Strong experience on data structure.
Skills and Qualifications
- Proficient understanding of distributed computing principles
- Proficient in Java or Python and some part of machine learning
- Proficiency with Hadoop v2, MapReduce, HDFS, Pyspark, Spark
- Experience with building stream-processing systems, using solutions such as Storm or Spark-Streaming
- Good knowledge of Big Data querying tools, such as Pig, Hive, and Impala
- Experience with Spark
- Experience with integration of data from multiple data sources
- Experience with NoSQL databases, such as HBase, Cassandra, MongoDB
- Knowledge of various ETL techniques and frameworks, such as Flume
- Experience with various messaging systems, such as Kafka or RabbitMQ
- Experience with Big Data ML toolkits, such as Mahout, SparkML, or H2O
- Good understanding of Lambda Architecture, along with its advantages and drawbacks
- Experience with Cloudera/MapR/Hortonworks
This job description provides a high-level review of the types of work performed. Other job-related duties may be assigned as required.
---
**Job Family Group:**
Technology
---
**Job Family:**
Applications Development
---
**Time Type:**
Full time
---
**Most Relevant Skills**
Please see the requirements listed above.
---
**Other Relevant Skills**
For complementary skills, please see above and/or contact the recruiter.
---
_Citi is an equal opportunity employer, and qualified candidates will receive consideration without regard to their race, color, religion, sex, sexual orientation, gender identity, national origin, disability, status as a protected veteran, or any other characteristic protected by law._
_If you are a person with a disability and need a reasonable accommodation to use our search tools and/or apply for a career opportunity review_ _Accessibility at Citi ( _._
_View Citi's_ _EEO Policy Statement ( _and the_ _Know Your Rights ( _poster._
Citi is an equal opportunity and affirmative action employer.
Minority/Female/Veteran/Individuals with Disabilities/Sexual Orientation/Gender Identity.
**Responsibilities:**
+ Utilize knowledge of applications development procedures and concepts, and basic knowledge of other technical areas to identify and define necessary system enhancements
+ Identify and analyze issues, make recommendations, and implement solutions
+ Utilize knowledge of business processes, system processes, and industry standards to solve complex issues
+ Analyze information and make evaluative judgements to recommend solutions and improvements
+ Conduct testing and debugging, utilize script tools, and write basic code for design specifications
+ Assess applicability of similar experiences and evaluate options under circumstances not covered by procedures
+ Develop working knowledge of Citi's information systems, procedures, standards, client server application development, network operations, database administration, systems administration, data center operations, and PC-based applications
+ Appropriately assess risk when business decisions are made, demonstrating particular consideration for the firm's reputation and safeguarding Citigroup, its clients and assets, by driving compliance with applicable laws, rules and regulations, adhering to Policy, applying sound ethical judgment regarding personal behavior, conduct and business practices, and escalating, managing and reporting control issues with transparency.
**Qualifications:**
+ 3 to 5 years of relevant experience
+ Experience in programming/debugging used in business applications
+ Working knowledge of industry practice and standards
+ Comprehensive knowledge of specific business area for application development
+ Working knowledge of program languages
+ Consistently demonstrates clear and concise written and verbal communication
**Education:**
+ Bachelor's degree/University degree or equivalent experience
This job description provides a high-level review of the types of work performed. Other job-related duties may be assigned as required.
Additional Job Description
We are looking for a Big Data Engineer that will work on the collecting, storing, processing, and analyzing of huge sets of data. The primary focus will be on choosing optimal solutions to use for these purposes, then maintaining, implementing, and monitoring them. You will also be responsible for integrating them with the architecture used across the company.
Responsibilities
- Selecting and integrating any Big Data tools and frameworks required to provide requested capabilities
- Implementing data wrangling, scarping, cleaning using both Java or Python
Strong experience on data structure.
Skills and Qualifications
- Proficient understanding of distributed computing principles
- Proficient in Java or Python and some part of machine learning
- Proficiency with Hadoop v2, MapReduce, HDFS, Pyspark, Spark
- Experience with building stream-processing systems, using solutions such as Storm or Spark-Streaming
- Good knowledge of Big Data querying tools, such as Pig, Hive, and Impala
- Experience with Spark
- Experience with integration of data from multiple data sources
- Experience with NoSQL databases, such as HBase, Cassandra, MongoDB
- Knowledge of various ETL techniques and frameworks, such as Flume
- Experience with various messaging systems, such as Kafka or RabbitMQ
- Experience with Big Data ML toolkits, such as Mahout, SparkML, or H2O
- Good understanding of Lambda Architecture, along with its advantages and drawbacks
- Experience with Cloudera/MapR/Hortonworks
This job description provides a high-level review of the types of work performed. Other job-related duties may be assigned as required.
---
**Job Family Group:**
Technology
---
**Job Family:**
Applications Development
---
**Time Type:**
Full time
---
**Most Relevant Skills**
Please see the requirements listed above.
---
**Other Relevant Skills**
For complementary skills, please see above and/or contact the recruiter.
---
_Citi is an equal opportunity employer, and qualified candidates will receive consideration without regard to their race, color, religion, sex, sexual orientation, gender identity, national origin, disability, status as a protected veteran, or any other characteristic protected by law._
_If you are a person with a disability and need a reasonable accommodation to use our search tools and/or apply for a career opportunity review_ _Accessibility at Citi ( _._
_View Citi's_ _EEO Policy Statement ( _and the_ _Know Your Rights ( _poster._
Citi is an equal opportunity and affirmative action employer.
Minority/Female/Veteran/Individuals with Disabilities/Sexual Orientation/Gender Identity.
This advertiser has chosen not to accept applicants from your region.
8
Big Data Developer
Bengaluru, Karnataka
Impetus
Posted today
Job Viewed
Job Description
Job Description:
Experience in working on Spark framework, good understanding of core concepts, optimizations, and best practices
Good hands-on experience in writing code in PySpark, should understand design principles and OOPS
Good experience in writing complex queries to derive business critical insights
Hands-on experience on Stream data processing
Understanding of Data Lake vs Data Warehousing concept
Knowledge on Machin learning would be an added advantag
Experience in NoSQL Technologies – MongoDB, Dynamo DB
Good understanding of test driven development
Flexible to learn new technologies
Roles & Responsibilities:
Design and implement solutions for problems arising out of large-scale data processing
Attend/drive various architectural, design and status calls with multiple stakeholders
Ensure end-to-end ownership of all tasks being aligned including development, testing, deployment and support
Design, build & maintain efficient, reusable & reliable code
Test implementation, troubleshoot & correct problems
Capable of working as an individual contributor and within team too
Ensure high quality software development with complete documentation and traceability
Fulfil organizational responsibilities (sharing knowledge & experience with other teams/ groups)
This advertiser has chosen not to accept applicants from your region.
9