Some news: I will be speaking at the American Chemical Society’s 239th National Meeting & Exposition, March 21-25 2010 in San Francisco, California.

Specifically, I will be speaking as part of the Future of Scholarly Communication symposium, which is hosted by the Chemical Information Division (CINF) of ACS.

As you can guess, the symposium is one of many such efforts at coming to grips with the potential of Web technologies to ameliorate how researchers use the literature, whether in terms of access, comprehension, search specificity or machine usage. So far, I have to say we’re still very much in the “coming to grips” part of the problem, which I guess is to be expected, given the recentness and vastness of the possibilities offered by Web technologies.

This division consists of librarians, publishers, software and database vendors, and scientists in the fields of chemical information, chemical informatics, and drug discovery.

My preliminary title is “Putting the Researcher Forward: Expertise Databases For Better Navigation of the Literature”, and will likely focus on use cases whereby a database of semantically robust metrics can be facilitate the relationship of scientists with the literature. I’ll probably try to address cross-disciplinary issues as well, although that’s a tough one.

Calit2 is a general public, visually pleasing expertise finding system that incorporates now-classic Web 2.0 techniques such as tag clouds and AJAX. It is focused on researchers associated with the California Institute for Telecommunication and Information Technology, basically scientists at the University of California San Diego (UCSD).

Launched in 2007, it covers more than 650 researchers and does a nice job of being informative without being overwhelming. However, like all such academic systems that I know of, it doesn’t provide comparative information of the sort that would enable differentiating scientists in terms of research domains, degree of expertise, or scientific prominence (aka “research impact”).

For example, as of this writing it lists16 scientists involved in bioinformatics. Great stuff, but you can’t easily distinguish their specific research areas within bioinformatics (yes, a tough task indeed), nor generate a report that would give you a sense of how influential each researcher in terms of numbers of papers, grants and grant dollars, patents, etc.

As I said earlier, the latter limitation is typical of academic systems, presumably because their researchers might not appreciate being ranked in such an explicit manner, even though this is precisely what happens when they are evaluated for tenure, and to a lesser extent, when applying for funding. And of course, we’re just applying the GPA principle, though one could argue that one’s GPA isn’t being displayed for all to see and that the number is calculated when one has to join a school, admittedly not the case here…

This is one reason we provide a ranking of researchers using our GOPR score. Since we’re not beholden to any particular research institution, and although we’re trying to be sensitive about it, ResearchScorecard can afford to take the risk of annoying some of our fellow scientists, or at least as long as our ranking system makes sense and is operating correctly. I hasten to say that ranking scientists (or any professional) in a rigorous and fair manner is difficult indeed. We’re definitely not done here, and so we’re always very interested in hearing suggestions and comments from our, ahem, “research subjects”.

And if you would like to raise your ranking, it’s actually very straightforward to do so. All one has to do is to get more grants, publish more papers and obtain more patents, among other things. Hey, I didn’t say it would be easy…

Many sectors of American society like to dump on the federal government. I often disagree as to the pertinence of these criticisms. Rather, I frequently observe amazingly smart initiatives and accomplishments, close to miraculous given how large an organization we are talking about.

Here’s an example: Applying the principle of search motivation to connect individuals who may have valuable information to share. Called iHarvest, it is being developed for the Department of State so that government employees who are researching similar individuals can discover that others are doing the same. That very observation might be highly meaningful if one party has bits of information the others don’t.

Yes, there are all sorts of knowledge management issues here. E.g., what if no one has any “proprietary” information? Even so, there is value in having the parties come together to realize that they don’t know any more as a group than they do individually. Remember, the beginning of wisdom involves understanding the limits of one’s knowledge.

Now, where might have you heard of this business of using the search motivation to connect X with Y? Hum, perhaps…Google! Yup, that’s the core of the Big G’s business model right there, now being applied for matters of security.

And oh by the way, this was brought to my attention by a monitoring agent of the government’s impressive FedBizOpps.gov repository of business opportunities, all for free, though you will likely need an account to access the link to the description of iHarvest.

Below I’ve highlighted the significant bit from the project description, just to spare you reading the required turgid governmentese:

The Department of State (DOS), Bureau of Diplomatic Security (DS) has an unusual and compelling need for immediate support for a unique iHarvest capability that leverages new information technology to automatically build user models based on analyst or operators activity and interests. This capability will automatically alert DS personnel to the fact of other individuals within DS that are conducting similar research or analysis and connect both parties. Additionally, the capability will support connections outside of DS with other interagency partners as DOS embraces a Whole of Government approach. In order to transform the enterprise of DS into an interagency compatible organization, there is an immediate need for greater data discovery among our intelligence partners and within DOS writ large, and this capability is an immediate first step to address this need. Particularly, as US Department of Defense forces reduce their presence in Iraq the DS agents immediately require an automated mechanism for sharing information amongst themselves and interagency partners. The capability will plug-in to DS existing situational awareness systems that support intuitive spatial interaction (Google Earth). Without this capability, the Departments ability to conduct diplomacy and business in high threat areas and around the world may be at risk which could affect the Departments mission. Further, it would impair the Departments ability to support national security requirements. Vital pieces of information that one individual is working with could go undiscovered by an office (or agency) that is involved with the same problem set. DS personnel are in danger at these high threat areas if the protective services personnel are not provided with this capability there exists a grave danger for their personal injury as well as injury to the individuals they are assigned to protect. The objective of this activity is to provide iHarvest integration research, design, development, integration, fielding and technical review support to the DS office: establish an alternate services and operations center for integration, operational testing, and evaluation. This information center will be an intricate part of a network of agencies where personnel conduct multi security level intelligence, law enforcement and counterterrorism operations.

Very cool, and great idea. My hat is off to the nameless bureaucrat(s) responsible for getting this off the ground. Who says government is necessarily lacking in imagination? Not I…

Here at ResearchScorecard we love numbers, especially when it concerns understanding funding decisions and projected impact on future research and product usage. The first batch of ARRA awards has now been released and we’ve been analyzing where the dollars are going.

Table 1	We first looked at the distribution of funds to the six universities we cover currently (Table 1 and 2).   Although UCLA received the largest amount, on a dollar-per-principal investigator (PI) basis it ranks second, with a far smaller average-dollars-per-PI compared with the number one per-capita recipient, The Scripps Research Institute in La Jolla, CA ($355K vs. $270K per PI).
Table 2	To explain this sizeable difference, I postulate that grants to Scripps researchers tend to be more clinical in nature, since clinical research is notoriously more expensive than non-clinical research. I confess I haven’t dug deep enough to confirm this, though.
Figure 1	UCLA also comes in first with the largest number of grants awarded per recipient PI (Fig 1). This doesn’t mean a whole lot, though, as UCLA is substantially larger that ther other institutions in our database, and so you would expect them to score highest in this respect simply because they have more researchers. However, Caltech and Stanford are noteworthy for tying in second place for the number of awards, since both institutions are much smaller than the others.   I believe this reflects the outstanding average quality of the research performed at these institutions, as measured by scientific impact, grant funding per PI and other factors.
Figure 2	When ARRA awards are analyzed with respect to the primary research area of the recipients, researchers involved in immunology, computational biology and genetics scored best in securing ARRA funding (Fig 2).
Table 3	Table 3 lists the specific primary research topics of the PIs that make up the summary areas in Fig. 2, along with the matching funding, and oh by the way, this classification accounts for 68% of the funding, as we don’t have data on some of the recipients at this time (won’t last long, not to worry).
Figure 3	Another interesting analysis asks the question “does the funding go primarily to oustanding researchers?” The short answer is no. Correlating the distribution of awards with researcher ranking based on our GOPR metric, we find that recipients are broadly distributed, though there is a small cluster of recipients of multiple ARRA grants in the high percentile range (Fig. 3). Interestingly, the sole individual that received three grants doesn’t score very high.
Table 4	Finally, Table 4 looks at trademarked product usage for a dozen products that recipient researchers have mentioned in their papers since 2006.   I’m also including a comparison of usage between researchers whose GOPR was in the top 10% in 2008 (i.e., 90% percentile) and all recipients. Comparing with the 90% GOPR percentile, products that are significantly under-utilized are listed in green, whereas products that are significantly over-used are listed in orange.

We’ve been busy in the last couple of months, with the result that many improvements and new functionality have been added to ResearchScorecard.

First, a login is NO LONGER NECESSARY for most reports. However, you’ll still need to login for making purchases and accessing certain kinds of information, such as a viewing researcher’s grants awarded within the last six months.

Second, a boatload of new functionality has been introduced:

1.  Product Usage report. This report, unique to ResearchScorecard, enables a quantitative analysis of trademarked products used by academic bioresearchers, as reported in the scientific literature. I’ll be writing more on that nifty tool in a later blog entry. Try it here.
2.  
3.  BuddyGrant notifications. We’re very proud of this one! As the name implies, this tool notifies registrants of grants newly awarded to one’s collaborators. As far as we know, this is also a ResearchScorecard exclusive [more].
4.  
5.  Collaborator Network report. This new (well, kind of new) report displays the collaborators of a researcher as known to ResearchScorecard. Remember, we don’t cover all institutions and all time periods, so yes, we are missing some collaborators. This functionality is also available via the Researcher Profile report, but we figured it made sense to make it available on its own as well. Try it here.
6.  
7.  Custom reports. We’ve been generating all sorts of nifty custom reports for Life Sciences vendors interested in developing better sales leads and improving their market awareness. This is another topic I’ll be writing about shortly.

Interesting case study of POPS produced by Clark & Parsia , a semantic web firm.

POPS is a NASA expertise location system which aims to “integrate NASA’s information about its nearly 70,000 combined civil service and contractor workforce in one place, linking the relevant, related information to form a comprehensive data service for staffers, workforce planners, analysts, and related personnel.”

POPS makes use of semantic Web technologies such as RDF to integrate data which are delivered via jSpace , is a visual query builder and Linked Data browser for SPARQL and other RDF query languages.

I particularly like their social network visualizer and its ability to overlay skills on top of the familiar “who-has-worked-with-whom” network (fig. 2 in the white paper), though it does look like an awful lot of navigation may be required. I also wonder about how much detail can be overlaid unto the network. Still, very nice work.

POPS' Social Network plugin

This post is really a post-scriptum to the previous blog entry, namely, ResearchScorecard now provides full-functionality coverage of the University of California, San Diego (UCSD):  Expertise Searching, Researcher Ranking, Comprehensive Researcher Profile, Funding Profile, Departmental Profile — it’s all working.  It’s also quite instructive (more on that later).

Although we’ve been quiet about it, ResearcherScorecard users may notice that a broader range of universities are now monitored. In the example below, you can see many more Life Sciences research institutions than what we’ve provided so far (Stanford and UCSF), including institutions that aren’t universities.

What’s missing presently are some of the comprehensive reports which are currently limited to Stanford and UCSF, especially the Comprehensive Researcher Profile (Tools → Researcher By Name) and its institutional counterpart (Tools → Organization By Name). However, very shortly we’ll be adding such reports, providing coverage way beyond Stanford and UCSF. First up will be UCSD, will many more to follow.

Example of broader institutional monitoring by ResearchScorecard

I’m pleased to announce that the latest GrantCard report for NSF biomedical fundees at Stanford is now available. As always, NSF reports for Stanford and UCSF are offered free of charge and can be accessed via Tools/Create sales lead report. An account (also free) is required.

Stanford University NSF grantees for week of April 13 2009

It’s interesting to see how few UCSF NSF fundees there are, compared to Stanford (see table below), even though UCSF is a much larger institution than Stanford. This is why UCSF receives far more funding overall in the life sciences, even though in 2008 the average funding per PI was $147,080 vs $32,789 for Stanford and UCSF, respectively, according to ResearchScorecard’s data.

This difference in NSF funding presumably reflects the greater extent of basic research life sciences research at Stanford, whereas UCSF is presumably more clinically oriented. I emphasize the presumptive nature of those statements because I must confess that I have yet to scrutinize the nature of biomedical research between the two institutions, but when I do I’m sure I’ll find all sorts of interesting differences.

I’ve always been impressed with the research sponsored by the Defense Advanced Research Projects Agency (DARPA), and as of today ResearchScorecard displays life sciences researchers at Stanford that have been funded by DARPA after 2000.

Granted, there aren’t many of those in the bio realm (see bottom table). So why bother? Well, it’s safe to say that these scientists are likely to become trendsetters if they aren’t already. This is based on the notion that DARPA will fund you because your project is both ambitious enough AND just might work, a combination that is typically difficult to achieve with conventional funding. This is because DARPA doesn’t work via a review committee as do NSF or NIH. Instead, you have to get approval from a program manager (a very senior and experienced scientist) to submit a proposal following a DARPA RFP. This proposal has to convince the program manager, who then has to convince the head of DARPA for things to move forward. As long as the administrators are smart and schrewd, this can lead to surprising results, as this process diminishes barriers to creativity often encountered when trying to convince committees (the more folks to convince, the more likely one of them will object to your proposal). And sure enough, the results speak for themselves: the Internet, desktop graphical user interfaces, stealth, GPS and cars that drive themselves are several among many examples.

Note that the dollar amounts for those grants or contracts aren’t listed in our database, as the amounts are not easily decipherable. Furthermore, I don’t believe the number is critical for the goal here: The imprimatur given by DARPA is sufficient to indicate that the recipient is a markee scientist, in my opinion. Let us know if you think otherwise.

DARPA funding data can be found in Researcher Profile and Department Profile reports for biomedical researchers at Stanford University for 2001 onward.  Use Tools Researcher by Name to get details for individual scientists. You can also use Tools Funding History.