As a news junkie, it's not surprising that one of my favorite APIs is newsapi.org, a site that supposedly gives you the ability to "search for articles from over 30,000 news sources and blogs." I have no idea if 30,000 is accurate but they certainly do bring in articles from all over the planet, and in multiple languages.
This site works like many others in that you enter a specific URL with search terms built into it. The search string/URL below begins with http://newsapi.org/v2/everything. Best I can tell, newsapi.org/v2/ is the base for al searches. The next part, everything, is one of two main endpoints. The other is top-headlines. For example, http://newsapi.org/v2/everything and http://newsapi.org/v2/top-headlines. For this lesson we will stick with everything.
http://newsapi.org/v2/everything?q=bernie&sanders&from=2020-02-26&sortBy=publishedAt&apiKey=f9e0d34612564433864a1d6495b703e6
The next part, q=bernie&sanders are the search terms, followed from which controls the date, sortby which insures that the very latest article rises to the top of your page and apiKey.
In order to make this resource truly useful, we need to arrange things so that search terms can be entered into a form to create a functional url that will retrieve data. At that point, the data can be parsed into headlines that are linked to actual articles.
- Create a new html document along with an attached javascript file or a script section within your html document.
- Visit newsapi.org and get your very own apikey.
- In the JS section, break the url into pieces that you will reassemble later.
- Set the base url as in let url = "http://newsapi.org/v2/everything?q=";
- Create an empty string for your first search term as in: let searchTerm = "" . Later on we will tie searchTerm to the value of a text field.
- Create and modify a date object so that it will capture today's date in the correct format for the ultimate URL. This part is tricky but, luckily for you, I have already figured it out.
- Create a new date object like this one: var dadate = new Date();
- On the next line modify the date object like this:
dadate = dadate.toISOString() - Console.log dadate as in console.log(dadate); and take a look at the output which should look much like 2020-02-26T22:34:50.092Z . As you can see this provides the date in the correct format, but also appends a lot of additional stuff. That extra stuff is the actual time in Greenwich England broken down into microseconds! But we don't need all of that. In fact, we only need the first 10 characters of that date string.
- To get rid of the extra stuff, add this line:
dadate = dadate.substring(0,10);
This handy little command sets the length of the string to 10 characters, and everything after the 10th character disappears. Do another console.log and you will see that we have a perfectly usable 10 character date string that will update every time our page is refreshed!
- Create another variable for the sortby portion of the url as in :
let sort = "sortBy=publishedAt"; - Create an apikey variable like this one:
let api = "&apiKey=AddYourKeyHere"; - Add an input text field like this one to your html:
<input type="text" id="search"> - Add an empty unordered list like this one to the html:
<ul id="articles"></ul> . Later on we will load this list with article titles. - Use the screen capture below as a model to test everything out. You will, of course, need to add a button to your html that calls the function, and enter a term into the search field.
- Using the search term "uga" I ended up with the following search string:
Your String
http://newsapi.org/v2/everything?q=uga&from2020-02-27&sortBy=publishedAt&apiKey=f9e0d34612564433864a1d6495b703e6
ORIGINAL String
http://newsapi.org/v2/everything?q=uga&from=2020-02-26&sortBy=publishedAt&apiKey=f9e0d34612564433864a1d6495b703e6which looks a bit different from the original string shown below.
Specifically, it is missing &from= between the search term and the date. So add that into your console.log statement as a string between those two values. Keep testing in console.log until you get it right. You can do this! - Following the model below (btw, you can save a lot of time if you copy resources from a previous project such as asteroids) to capture the number of articles that are generated by a search. You don't actually need this information, but it's a handy way to determine if your function works properly.
- Now that your function works, create a variable that captures your articles: dadata = data.articles; (for example)
- Next, add a loop that will append the title of each article to the articles id, along with appropriate <li> tags. BTW, you could do this with straight Javascript, but in this case I am using Jquery syntax which is a bit shorter.
- Test it out. Did it work? Great! Let's move on and create links to those articles. As you can see from yet another screen capture, I created a variable and integrated it into the append statement. Try it.
- As you can see, it works, kind of, but lots of those links do not work. Inspect those unlinked titles to try and see what is going on. When you do, you may see something like this:
For some reason, the link is closing without adding the title between the link and the closing </a> tag. When I encountered this problem it really bugged me and it took about an hour to figure out what was going on and to resolve the problem. If you return to your console and click on the link to the actual json feed you may be able to figure it out yourself. Do so, before you read the solution.Solution: So here's the deal. Some of the urls end with a forward slash. For SOME REASON that Javascript understands but I don't, that forward slash causes the link to close without adding the title. I googled the problem but never found anyone else who had mentioned the problem. So, in a deep geek state, I fixed it with a bit of string magic. Looking below you will see that after tying the variable urly to the url value, on the next line we created a variable named lastChar that captures the last character of the url string. Afterwards, we add a conditional statement that trims the last character from the string if it is a forward slash. And now all of the links work!
The Challenges
Challenge 1: Search twice without refreshing your page. Notice how articles continue to fill up the list without clearing it first? That's pretty unwieldy, so your job is to figure out how to clear that list before a lot of new stuff is loaded.
Challenge 2: So far we have created a search field that works for one search term but what if our users want to use multiple search terms such as Space Exploration instead of simply Space or Exploration? One solution would be for the user to enter space&exploration into the search field but that's a lot to ask of the average website visitor. We need an easier way. Your job is to find one.
Hint: I used a string property (. length ) along with a conditional statement to solve this problem WITHOUT changing the getJson statement. 
Challenge 3: This API's documentation at https://newsapi.org/docs demonstrates how to construct a url with search modifiers, including top-headlines and specific news sources, which means that there are more options for use to construct an even more robust search experience for our users. Take a look at these options and try to implement at least one of them. BTW, I could imagine this exercise morphing into a mid-term project if you find yourself enjoying all of this.