Almost every year I encounter  yet another industry veteran  engaged in spin doctoring a sorry database, pitching it to naïve customers,  presenting fiction as facts at conferences and on the exhibit floors, and not only selling snake oil but also concocting it.  As I am getting closer to that veteran status (or at least age), I am more concerned than ever, what changes aging may bring in my ethical judgments and behavior.

This year hopefully ends the 3-year tribulations caused by a database developed and hyped by two veterans of the information industry.

The idea of including the cited references from journal articles in psychology was excellent, but the other aspects (the journal base, the volume of updates, and the rock bottom pricing at half or third the price of PsycINFO, were utterly naïve and infeasible. The implementation and especially the promotion of this database have been appalling.   Such attitude gives  the information industry a bad name, and hurts not only its customers, but also those companies which have been playing fair game for a century,  such as the H.W. Wilson Company.

These plans and promises were unrealistic at a time when PsycINFO was abstracting and indexing about one third of e-psyche targeted journal base (without including cited references then), and had a loyal and very wide user base which grew up on the print version of Psychological Abstracts, then from the 1990s converted to one of the two dozen  CD-ROM or online versions of the most popular and most widely used commercial database. PsycINFO also had an experienced indexing staff and was backed by the American Psychological Association in more ways than one.

What PsycINFO did not have at that time was promised to be the highlight of e-psyche (that I myself applauded in testing the pilot version which had complete sets of cited references, while criticizing the unrealistic journal base): the inclusion of the “cited references or citations as found in the original source”. The Institute for Scientific Information (ISI) has done that for about 500 psychology and psychiatry journals at that time, creating about 35-36,000 records with nearly 1 million citations from these journals every year, going back several decades. Had the creators of e-psyche asked Eugene Garfield about the process and the expenses, they could have stopped their not exactly innocent daydreaming and relentless bamboozling of  the potential customers (and the press) by presenting fiction as facts. 

 THE SOURCE BASE

The first public document which I found to mention the incredible journal base of e-psyche is the talk of Dennis Auld at the ASIDIC 2000 conference. As of the end of December, 2003 there were records from less than 1,200 journals (very sporadically selected). There is not a single record in e-psyche about books, there is not even a document type for books. Neither appears in the publication type index any entry for archives.

There are 20 records about web sites, and 401 records for pre-prints, all of which are for the journal Psycoloquy. These are, by the way,  not preprints as the journal does not have a print version, being one of the Web-born and Web-only psychology journals. 

Judging from the target journal list, in dreaming up this 4,200 journal base, the managers of e-psyche were desperate to find any warm body which may seem to be related to psychology and behavioral sciences even if they aren’t, such as the dubious quality e-zine Lies People Tell and a great many other fluffy magazines that not even Dr. Phil’s fans would touch. Actually, it did not  matter if the bodies were not even warm, such as the titles which ceased publication before the start date of e-psyche (such as Laughing Matters which was last issued  in 1990), or scholarly ones  which were announced but never published, such as Psychological Studies in Africa.

The absurd figures about e-psyche must have been mentioned so often in talks with potential customers, investors and journalists, that they started to be treated as reality, and the emperor’s cloth was colorfully described on many university sites which had the misfortune to license the database. The blurb on  one site pumped up the journal base to 5,000.  In a rather scripted  interview given to ACCESS, president John Kuranz asserted in March 2001 that “e-psyche includes over 4,000 journals and PsycINFO about 1,500”.  Gary Condit would have given his congressdom if he could have gotten away so easily with his answers to Connie Chung in the memorable interview.

 

Encouraged by the interviewer’s docility, a few paragraphs later president Kuranz  further elaborated on this issue, and the journal base grew larger, to 4,260, expressed in more precise measures, and with seriously sounding analysis. The statement about overlap  sounds convincing, impressive and scholarly, but it is not true as e-psyche had these 4,260 journals only in their managers’  fantasy, so the rest of the comparison is as accurate as Baghdad Bob’s statements were in his press conferences.

Interestingly, a year later, in response to the excellent critical review in Online Mitteilungen by Eveline Pipp,  president Kuranz downgraded  his journal base significantly to 857 from his own earlier claim of 4,260. Even his unverifiable reference to the 1,390 purportedly contracted journals is less than a third of what he asserted earlier saying that “we cover 4,260 journals” or   “e-psyche includes over 4,000 journals”. What matters for the users is how many journals are processed (and how comprehensively), not how many are contracted, talked about or dreamed about.

CITED REFERENCES

 Inclusion of cited references back to 1970 was the greatest promise, and the greatest ruse in selling this database. Excluding references citing pre-1970 publications is somewhat strange but one can live with it. However, the exclusions which are not mentioned but heavily practiced are unacceptable. Consistent depth of coverage in e-psyche in any regard is the ultimate oxymoron. Consistency in  inclusion of cited references is a joke, and a very bad one at that. The “if available” clause, as much of the text  is just presidential pompousness and pseudo precision.

For appropriate context, the chart shows what percentage of the records in e-psyche and in the psychology, psychiatry and behavioral sciences subset of the ISI Social Science database  in the Web of Science (WoS) have cited references from 1996 onward. It is normal that about 20% of the journal items  (and their records) have no cited references.

PsycINFO promises comprehensiveness for inclusion of cited references only since 2001, and it delivers, although not perfectly. In the ISI Social Science component of WoS comprehensive inclusion of cited references is implied, and  in e-psyche it is explicitly promised (except for pre-1970 cited refs). Still, almost half of the records in e-psyche in 2001 and 2002 had no cited references at all, and by 2003 more than 80% of the records were added a single cited reference. 

As for the total number of cited references it cannot be determined for e-psyche because it does not have a field which would provide the citation count (ISI and PsycINFO have that information). But this quote below from president Kuranz reveals that the average number of citations per record is 9.93 in e-psyche. Even if we use as denominator the number of records which had cited references (a meager 55.4% of the total) in e-psyche, instead of the total number of records,  in May, 2002 the ratio was still less than 20 cited references/record. In the psychology/psychiatry subset of the ISI Social Science database this figure is 33, and in PsycINFO it is 44 (because it indexes books which have much more cited references on average than journal articles).

This miserably low rate of cited references/record may not be obvious when you look at the e-psyche record by itself. You may as well believe that there are only 10 cited references in the article.

But if you have access to PsycINFO (or to the article), and see that the article has 54 cited references (and PsycINFO displays all of them), it becomes very apparent how much e-psyche shortchanges the users. Not for this single record but for almost all the records in its database – way beyond the questionable exclusion of citations to books and any pre-1970 journal articles. 

In my summarizing spreadsheet excerpt you can see side by side which cited references are included in only PsycINFO, and in both databases. There is no rhyme or reason for how the 10 citations were selected by e-psyche.

Of the 54 cited references, 38 are for journal articles, and 36 of those were published in 1970 or later. Even by e-psyche’s own inclusion rules 36 cited references should have been included. The never published policy of limiting the inclusion of cited references to 10, makes a farce of the claim that e-psyche “include[s] all bibliographic reference citations, if available, back to 1970. 

This pattern is clear also from the summary of the number of cited references for the articles published in the 2002 volume of Psychological Inquiry, a top ranked psychology journal. PsycINFO has a total of 1,955 cited references for this volume, e-psyche has 439. It did not help e-psyche that for more than half of the articles it did not even have a record You may consult the entire table here

Concerned over losing the chance of getting a slice of the German market, after the critical review which accurately confronted him with the facts, president Kuranz fessed up a bit in a long letter, and made some new promises, which turned out to be  like those of the desperate debtors who are long on promises and short on delivery. Very, very short in this case.  Beneath the nice round numbers, there is the indication to further emaciate the cited references to about 6.7 per record: 1 million citations spread over 150,000 records.

DATABASE SIZE AND UPDATE FREQUENCY

Well, as of December 2003, the reality is far more discouraging and appalling. If you count the journal name entries by browsing the journal index (the only reliable way to know the number of journals really covered (no matter to what a narrow or broad extent), you will end up with 1,182 journals as of December, 2003. That is barely more than a quarter of what was claimed two years earlier, and less than half that was promised for March of 2003.

As for the 150,000 record count also promised by March, 2003,  there were 113,459 records in e-psyche by the end of December. This number includes the duplicates, such as the dozen ones I spotted in a single volume of  Annual Review of Psychology alone. Notice the big difference between the cited references: the first items in the record pairs were created for the first batch of records at the start, the second items were created after the 10-citatiosn limit was introduced.

The monthly updates never reached even the half-mark of the promised 6,000 records/month. Even after reconfirming  this number in the letter by president Kuranz, the update pattern has not improved. It became worse, averaging 2,300 records/month, and ended in July, 2003. For comparison,  PsycINFO managed to add an average of  6,465 records per month, and its database is approaching the 2 million record mark with 10 million citations in more than 210,000 records. It made its own big gaffes and earned my jeers, but it never tried to mislead its customers with false claims and gross misrepresentation .

As for the currency of e-psyche, suffice it to say that there are only 1,565 records about articles published in 2003  at the end of the year while PsycINFO added 50,579 such records (and have one more update coming as I write  this review). 

As for cited references, in e-psyche less than 18% of the  records for publications in 2003 had cited references – almost all of them less than 10. Obviously, it was yet another move to save money. This is no surprise as there were five updates out of the seven of e-psyche when not a single record was added with cited references. For perspective, PsycINFO  had citations for more than 80% of the records created for documents published in 2003. 

The managers of e-psyche have taken for a ride three online service providers which host this embarrassing database, the libraries, the end-users, and fellow producers of databases (such as the Ulrich’s Periodical Directory and The Serials Database which report the nonsense claims of journal coverage by e-psyche spreading the myth).

I wonder if  NFAIS members would discuss this issue at their February conference, and if some of those users who were badly misled would form a class to test how limited  really is the liability of Database Access Group, LLC. I would love to represent them in the court of law, but I am not licensed to practice law, and I am not enough of  a veteran to pull this off, but I would chip in pro bono, if someone takes up the case.

Back to Cheers & Jeers 2003