« Comment Spam Killer? | Main | It's On »
News Archives?
Does anyone know a good resource for archived news stories? I'm looking for something like the Common Dreams News Center where archives are categorized by date and you can scan all the headlines from a given month or date range. In fact, the Common Dreams site is exactly what I'm looking for, but the selection of stories is too limited. Lexis offers archives of wire service stories and major newspapers, but the only way I know to reach them is through a targeted search and since I only vaguely know what I'm looking for during a certain time period I just want to browse headlines in that period. Many newspapers offer archives of their news stories, but they want you to pay for access and, again, the only way to reach the archives is through keyword searches rather than by date ranges. So, if you know of a resource where AP or Reuters or Knight-Ridder wire stories are archived by date range (like a blog would do it), please send me a link. Please? Pretty please? You'll be doing me a big favor! p.s.: This makes me think it would really be worthwhile for someone to start a blog where they just copied all the headlines from a major newspaper (or maybe BBC news—someplace where the archives are freely accessible) into a day's post w/links to the articles. Then, when people have research needs like mine, they could go to that blog, find the appropriate date(s), then scan the headlines for that date and access the articles they wanted. I bet this is out there, somewhere, I just don't know where...Posted January 19, 2005 04:06 PM | life generally
through my monkey news project (link), i have collected about 2 years of news headlines and blurbs from over 50 sources. i can put it all in an SQL database for you. sadly i have nothing that can host such a db at this point. i'd use something like postgres with a PHP interface, so if anyone wants to helps out ...
one of the major problems with this is that the AP only creates links that are ephemeral, so when you go back even a day or two later the story is gone. and, as you noted, some sites don't give access to their archives beyond 2 weeks.
however, if i can help, let me know.
Posted by: jose at January 19, 2005 07:22 PM
Hmm. How much server space would you need for that database? And would you be able to set it up so I could set date parameters and see all headlines between those dates? We might be able to set it up around here somewhere, at least on an experimental basis. I wonder if we could use BBC feeds instead of AP feeds. I think the BBC archives are stable and links remain open. The NYTimes has a way to access old archive links for free, too, but I'm sure that will be shut down before long. It would almost be worth getting a fund together to buy an AP subscription so we could archive our own AP stories in blog-like, easily-accessible format. I don't understand why bloggers haven't done this already. Ok, I do understand -- it would be costly and the AP would probably be pissed. I wonder if the AP's subscription contracts force users to agree *not* to make the archives accessible for free. That would be awesome. Even the news of the world is private property.
Hey, and what happened to your Mac news aggregator? It hasn't updated in a week for some reason.
Posted by: ambimb at January 20, 2005 08:58 AM
last things first: the macnews aggregator is dead. i didn't think anyone was watching ... the machine that hosted all of the aggregators (blogs, news, mac news, infosec news, etc) died, and i had to scrape the data off of the old machine. i forgot abotu the mac news aggregator .. i can go back and get it, but what i think i will if i grab it is to post the URLs to the sources and let people have it for themselves.
the disk space required is probably a gig of disk space. right now i'm securing someone's machine to do this. the web interface shouldn't be too hard to do, it just wont be pretty :). usable, but not pretty. feeds gathered include: AP, NYTimes, WP, BBC, guardian, boston globe, reuters, USAToday, MSNBC, LA Times, and many, many others.
what i want for search parameters is: date (start, end), source, key terms. display would spit out the timestamp, headline, source, blurn, and link to the story.
sound good?
Posted by: jose at January 20, 2005 11:22 AM
Sounds pretty cool, actually. I can't wait to see it!
Posted by: ambimb at January 20, 2005 08:44 PM
You can do this abusing your free Lexis/nexis account-- just do a search within a news group or publication using date limiters but leaving the search terms blank.
Posted by: Andrew at January 22, 2005 10:50 AM
