Online media monitoring helps businesses to build their brands and manage the way their products and services are being perceived in the market. This article gives a short tutorial on how to build an automated online media monitoring system using Node-RED.
online media monitoring refers to the process of scanning open or public data on digital platforms by looking for keywords on a particular topic. It helps you to manage the way your product or service is perceived in the market. Even though many premium media monitoring tools are available and come with a free trial period, the ‘feedparser node’ provided in Node-RED makes it an advisable alternative to the commercial ones. This node monitors an RSS (really simple syndication) or Atom feed for new entries. RSS refers to simple text files that contain updated information like news pieces, articles and dialects. An RSS feed first takes the headline and summarises it. Then it updates notices, and links them back to articles on the required website. This Web feed allows users and applications to access website updates in a standardised, computer-readable format. Figure 1 shows the relationship between RSS and Node-RED.
This article focuses on creating an automated online media monitoring system using Web APIs (application programming interfaces). This is done by incorporating a template node to design a user form that gets readers’ details. After the details are submitted, the system enables the reader to read the article. Using the feedparser node, this automated system is created for three RSS feeds for the popular news channels BBC, NDTV and CNN. From every feed, news on three verticals (such as sports, economics, and education) is filtered. The outputs are displayed in the Node-RED debug monitor and UI dashboard.
Procedure to construct and build the flow using Node-RED
- Install the latest version of Node.js from https://nodejs.org/en/download/
- Install Node-RED through the command window by giving the following command:
npm install –g node-red
- Through the browser, get access to the Node-RED flow using the IP address http://127.0.0.1:1880
- From the ‘Node palette’ of Node-RED, insert the appropriate nodes, configure their properties, and wire them.
- Confirm deployment by clicking on the ‘Deploy’ option.
- View the output through the ‘Debug’ monitor window and dashboard UI.
Online media monitoring – node flow design
- Install and add the feedparser node from Manage palette using the following command:
node-red-node-feedparser
2. Drag and drop feedparser node from the Node palette.
3. Browse the various RSS feeds of media companies like BBC, NDTV and CNN.
- Paste the RSS feed URL in the Feed URL box.
- Customise the refresh time if needed.
4. Connect the feed parser to a Text dashboard widget so that the XML is converted to JS object, and is displayed properly as text.
5. Add a switch node and configure the message.
6. Connect the feedparser node to switch node to change payload value to the news feed article value.
7. Type in the word you want to find in the switch box and set configuration to find regex (regular expression).
8. Click on the Ignore uppercase dialogue box to include matches regardless of the uppercase and add the change node.
9. The change node is to be configured in such a way that the msg.payload is set to msg.article.title. Set a no reply email in the msg.from object. This feedparser node is connected to a switch node and change node in order to be used as a news filter, i.e., to read a particular section of the news.
10. Apply the same rules for other news channels to add them with different titles or headlines like sports, education, and finance, and follow what was done above.
11. The template node is configured with HTML code to create a form that requires readers to sign in before they can continue reading. Connect the function node and the template node to create a media monitoring UI using HTML/CSS code and display the news articles. This template node helps in creating a form that asks for user details, which has to be filled in before viewing the news feed.
12. Insert a text node and set the data to be printed as the title of the article.
13. Insert debug nodes appropriately to view the output in the debug monitor. After creating the complete flow, click on ‘deploy’ from the top right corner of the window and deploy the system.
14. View the results in the debug console. To check each node’s output you can use node filters.
15. To view the user interface, go to the link 127.0.0.1:1880/ui/. Enter the required details and click on ‘Submit’. It will take you to the next page, where all the news articles can be read.
Inference
The feedparser node monitors RSS, Atom, and RDF feeds in Node.js. The polling time can be customised, though the default value is 15 minutes. Feed-parser is a transform stream operating in ‘object mode’: XML in -> JavaScript objects out. Every readable chunk is an object representing an article in the feed.
The flow in Node-RED for online media monitoring Property configuration
(i) Feedparser node: Three such nodes, each with different feed URLs are used in this flow.
- BBC: http://feeds.bbci.co.uk/news/world/us_and_canada/rss.xml;
- NDTV: http://feeds.feedburner.com/ndtvprofit-latest?format=xml;
- CNN: http://rss.cnn.com/rss/edition_entertainment.rss;
As the remaining configurations are the same, only BBC World News is shown in Figure 3.
(ii) Template node – dashboard widget: This is shown in Figure 4.
(iii) Switch node: The filter used is Matches regex, which is not case sensitive, as shown in Figure 6.
(iv) Change node: Three such nodes, each with different switch inputs, are used in the flow for BBC, CNN and NDTV, respectively, as shown in Figure 7. The same configuration is applicable for all.
(v) Text Dashboard node: Three such nodes, each with different feedparser inputs, are used in the flow.
- BBC: Belongs to webAPI tab BBC group; label: Now, new on BBC news
- NDTV: Belongs to webAPI tab NDTV group; label: NDTV is sizzling with.
- CNN: Belongs to webAPI tab CNN group, label: On CNN now!
Figure 7: Change node property configurations
Since the other configurations are the same, only NDTV News is shown in Figure 8.
Screenshots (outputs)
(i) The debug console output is shown in Figure 9.
(ii) The dashboard output has a Web user interface. This involves the creation of a Web interface, which consists of a reader’s login form to enter user details such as name, mobile number, and email ID (Figure 10) .
An RSS newsfeed dashboard with three different news sources and news categories is presented in Figure 11.
