The notebook contained in this project prepares, trains and saves a transformer based model (fine tuning a pre-trained distilbert) that will categorize the query into one of the following types:
-
- Retailer: Query is about a retailer (Target, Walmart, ...)
-
- Brand: Query is about a specific brand (Huggies, Gatorade, ..)
-
- Category: Query is about an open category (diapers, hand bags, phones, ...) The purpose of this model is to direct a query request to the specific treatment depending on the category. The output will be the predicted category and the confidence (in that prediction).
- The data is loaded along with the required libraries (transformers, torch, nltk...), the retailers are in 'offer_retailer.RETAILER', the brands in 'offer_retailer.BRAND' and the categories in 'categories.PRODUCT_CATEGORY and categories.IS_CHILD_CATEGORY_TO'.
- After cleaning the data, I realize that we need a lot more samples for 'categories', since it is the most 'generic' option and we would benefit from having a wide range of observations to feed the model.
- For this I semi-manually created a 'retail_product_nouns' list of potential 'category related words' and I enhanced that list by adding all synonyms of every word (using wordnet from nltk).
- Then the data was balanced (by upsizing the lower categories via bootstrap) and upscaled (by resampling with repetition on the resulting data)
- After this, the data is tokenized using a pre-trained BERT model (from HuggingFace) and loaded/transformed into a convenient format (torch tensors).
- Then, after a few iterations on parameters, I came up with a configuration that lets the model learn reasonably well.
- Finally, the model is saved for production.
**To run the jupyter lab file that trains the transformer with the exact same operating system, python version and required libraries in 'production' please run the jupyter lab through this Dockerfile:
docker run -p 8080:8080 -v "$(pwd)":/app/notebooks my_jupyter_lab_image
The complete deployment project can be found in this repository.