Skip to main content

Reproducing results from the text mining paper

The paper we are talking about is Data science for building energy efficiency: A comprehensive text-mining driven review of scientific literature

Work done in the original paper​

Work done by GQC​

  1. Captured various keywords related to water-networks from the ArcGIS website
  2. Categorizing the keywords in the categories of:
    1. data
    2. data-science
    3. phase
    4. water-networks
  3. Split compound words with a space (or split them individually)
  4. Use the pretrained GloVe embeddings trained on the Common Crawl Dataset
  5. Reproduce plots from the original paper using the method described in the paper.

Improving the work done by GQC​

  1. In the first iteration of the project, GQC used pre-trained GloVe embeddings which were trained on the Common Crawl dataset
  2. In the second iteration, we wish to train our custom word embeddings by scraping the papers from the Elsevier API and creating a model.
  3. The steps would be as follows
    1. Identify and curate a list of keywords obtained from the ArcGIS website
    2. Scrape papers using Elsevier API with those keywords (this will be done by Vannary)
    3. Have a text cleaning pipeline to clean and process the text
    4. Create a model to train the word embeddings (ideally GloVe, but word2vec, or fastText should work as well)
    5. and then re-create the results (plots) from the paper