Question 2

A large supermarket has released a dataset of all purchases made by its 10,000 customers after removing names, as “purchase-rest.csv”. The purchases are categorised into 40 items (numbered 1 to 40, inclusive). If a customer buys an item, the corresponding cell is 1, otherwise it is 0. You know that Bob is one of the customers. And you know his past buying habits (“bob.csv”) as background knowledge. You would like to identify which row in the released dataset is likely to be Bob’s data, thus re-identifying Bob. Note that there could be some items marked as 1 in bob.csv but marked as 0 in purchase-dense.csv in Bob’s row, and vice versa. Hence you cannot do a direct (subset) match as in Question 1. Instead, you seek to use a similarity metric, i.e., the Jaccard index (see Appendix).

(a) Write a program in Python that takes a list of binary values, and converts it into a set of indexes which are labelled 1 in the list. 

(b) Write a program in Python that takes two sets and computes their Jaccard index.

(c) Write a program that computes the Jaccard index of each row in the purchase-rest.csv dataset and your background knowledge about Bob, and outputs the top two rows with the highest Jaccard index.

(d) What are the row indexes, the set of items bought and the Jaccard index of the top two rows.