ARTH — Task 34
Task Description📄
🔅 Use Aggression Framework of MongoDB and Create Mapper and Reducer Program.
(Refer to examples at MongoDB official Documents)
What is MongoDB?
MongoDB is an open-source document-oriented database. MongoDB is not based on the table-like relational database structure but provides an altogether different mechanism for storage and retrieval of data, that’s why known as NoSQL database. Here, the term ‘NoSQL’ means ‘non-relational’
What is MongoDB Aggregation Framework?
Aggregation operations process data records and return computed results. Aggregation operations group values from multiple documents together and can perform a variety of operations on the grouped data to return a single result.
MongoDB — Map Reduce
In MongoDB, map-reduce is a data processing programming model that helps to perform operations on large data sets and produce aggregated results. MongoDB provides the mapReduce() function to perform the map-reduce operations. This function has two main functions, i.e., the map function and reduce function. The map function is used to group all the data based on the key value and the reduce function is used to perform operations on the mapped data.
Now its time to use the Aggression Framework of MongoDB and Create Mapper and Reducer Program.
The dataset which I am using contains First_name, Last_name, people(gender), and Country. Here we are going to count the number of people from each country. So, Now let’s import the dataset in MongoDB.
Let’s connect to the MongoDB shell:
Using the mongo command we can get into the mongo shell
crosscheck weather the dataset is imported or not using show dbs command in mongo shell
now you can check the collections… for now let’s directly jump to the another part of the task
Here we will perform this using two MongoDB Aggregation Framework.
- Aggregation Pipeline
- Map-Reduce Function
Let’s start with the first method — Aggregation Pipeline:
db.countries.aggregate([{$group: {_id: {country: “$country”}, no_of_people: {$sum: 1}}}, {$sort: {gender: 1}}]){$group: {_id: {country: “$country”} → group by people(gender-male-female)people: {$sum: 1} → count the total countries asscoiated with that people(male-female){$sort: {gender: 1} → sort them in ascending order
Map-Reduce Function — Method-2
MongoDB uses the MapReduce command for map-reduce operations. MapReduce is generally used for processing large data sets.
Declaring the Map variable:-
var mapFunc1 = function() {var cntry = emit(this.country, this.people);$split: [ cntry, “,” ];};
Above we have declared a variable that will be grouping the data of people and country name. Then splitting the data by commas.
Declaring the Reduce variable:-
var ReduceFunc1 = function(keycountry, valuespeople) {return valuespeople.length;};
We are counting the number of male+ female from the countries after the output has been sending by the mapper
Let’s use the MapReduce function
db.countries.mapReduce(mapFunc1ReduceFunc1,{out: “map_reduce”})
And saving the output of this function in map_reduced collection.
Let’s perform a query:-
db.map_reduce.find().sort( { } )The data is sorted and we have got what we wanted a number of people from each country.
