Journal of Biomolecular Techniques 17:200–206 © 2006 ABRF

AB RF

REVIEW The End of the Microarray Tower of Babel: Will Universal Standards Lead the Way? Ernest S. Kawasaki NCI Advanced Technolog y Center, Bethesda, MD Microarrays are the most common method of studying global gene expression, and may soon enter the realm of FDA-approved clinical/diagnostic testing of cancer and other diseases. However, the acceptance of array data has been made difficult by the proliferation of widely different array platforms with gene probes ranging in size from 25 bases (oligonucleotides) to several kilobases (complementary DNAs or cDNAs). The algorithms applied for image and data analysis are also as varied as the microarray platforms, perhaps more so. In addition, there is a total lack of universally accepted standards for use among the different platforms and even within the same array types. Due to this lack of coherency in array technologies, confusion in interpretation of data within and across platforms has often been the norm, and studies of the same biological phenomena have, in many cases, led to contradictory results. In this commentary/review, some of the causes of this confusion will be summarized, and progress in overcoming these obstacles will be described, with the goal of providing an optimistic view of the future for the use of array technologies in global expression profiling and other applications. Key Words : microarray, expression profiling, RNA standards, controls, MGED, MAQC, NIST, ERCC.

O

ctober 2005 was the 10th anniversary of a landmark paper on microarrays in the journal Science.1 The article described expression profiling of Arabidopsis genes via an array of cDNA fragments attached to glass slide substrates deposited by high-speed robotic printing. This was the first publication to describe the microarray method of measuring differential gene expression using two-color fluorescence hybridization. Following closely on this report was a study describing an entirely different array format that was manufactured by a photolithographic process on a substrate resembling a computer chip.2 In this case, 20-base oligonucleotides complementary to thousands of genes were synthesized in situ and used to estimate gene expression by measurements of hybridization employing a single-color fluorescence system. These two microarray platforms were the mainstay for expression profiling in the early years, but recently there has been a proliferation of array systems (Table 1) that, although different in detail, have basic similarities to

Address correspondence and reprint requests to : Ernest S. Kawasaki, PhD, NCI Advanced Technology Center, 8717 Grovemont Circle, Bethesda, MD 20892 (phone: 301-435-2891; fax: 301402-3134; email: [email protected]).

200

the original platforms.3 During the past decade there have been over 17,000 articles listed in PubMed when it is queried for “microarray” or “microarrays,” and the number of publications has been increasing almost exponentially, as illustrated in Figure 1. The great majority of these studies were done without internal controls or standards, so comparison of results in related research has been difficult or impossible. The pertinent differences in array platforms causing this confusion will be described, and some international efforts to alleviate the problems arising from this microarray “Tower of Babel” will be summarized. As this is not an exhaustive review, only a few publications will be cited for illustrative purposes, with apologies beforehand to the authors of many excellent articles not included. A Proliferation of Microarray Platforms and Associated Technologies

Table 1 gives a list of sources for obtaining whole genome arrays, which are defined as arrays that have approximately the entire gene complement of the genome represented on one slide or chip. You will note that there are large differences in the size of the probes, the number of probe sets, and the total number of probes per array. This and many other technological differences found in these platforms will be enumerated, with pointers as to how or why

Journal of Biomolecular Techniques, Volume 17, issue 3, July 2006

The End of the Microarray Tower of Babel

Ta b l e 1 Sources of “Whole Genome Array” Platforms Probe Size

# Probe Sets

ABI

Source

60mer

33,000

33,000

Affymetrix

25mer

54,000

1,000,000

Agilent

60mer

44,000

44,000

GE Amersham

30mer

57,000

57,000

Illumina

50mer

46,000

1,500,000

Microarrays, Inc.

70mer

49,000

49,000

NimbleGen

60mer

38,000

380,000

Phalanx Biotech Home Brew

# Probes/Array

60mer

30,000

30,000

50mer–n70mer or cDNAs

40,000

40,000

The probe size, number of probes sets and the total number of probes per array are indicated.

these differences can cause discordant results between platforms. The nomenclature convention followed here is that the “probe” is the gene sequence arrayed on the chip, and the “target” is the RNA sequence to be labeled and hybridized to the probes. Probe manufacture. The probes for the arrays may be made in situ by photolithographic or ink-jet methods, or by standard oligonucleotide synthesis protocols followed by attachment to various substrates.3 Because the methods are so varied, it is difficult to estimate the purity of the probes or their true sizes, and large differences in these parameters can have a great influence on signal intensi-

Figure 1

A decade of microarray publications. The number of publications per year derived from PubMed using the terms “microarray” or ‘microarrays” is shown.

Journal of Biomolecular Techniques, Volume 17, issue 3, july 2006

ties, background, hybridization kinetics, and so on. All commercial arrays are now oligonucleotide in nature, but many private or academic centers continue to print arrays with PCR products derived from cDNA clones. Probe substrates. Microarrays are made on several types of substrates, including a variety of activated glass slides, silicon chips, and membranes. These physical differences will have an influence on background, hybridization kinetics, and signal-to-noise ratios, again resulting in problems comparing data across platforms. Probe design and location. As oligonucleotide arrays are now the most common platform, probe design has become an important aspect of array technologies. Oligonucleotides commonly used in arrays can be 25, 30, 40, 50, 60, or 70 bases in length, a complicating factor in array design and comparative analyses. Probe sequences derived from areas of intense secondary structure in mRNAs may not be able to hybridize to their corresponding target. Most probes are derived from the 3′ end of the gene coding sequences to accommodate the fact that target labeling (see below) usually begins at the 3′ end of mRNAs. Probes that are derived from the 5′ end may not hybridize to targets that are made from the 3′ ends of the mRNA. Thus, depending on probe design, oligonucleotides from different platforms designed to assay the same gene can give widely different results. Probe size. It has been shown that larger probes (60mer70mer and cDNAs) produce, in general, higher signal intensities relative to the shorter variety when hybridized under similar conditions.4,5 On the other hand, the short probes may exhibit higher specificity under stringent hybridization conditions, but may have lower sensitivity

201

E.S. Kawasaki

for detecting mRNAs of low abundance than the long probe arrays. Thus, probe size can be a confounding factor when comparing the same genes across many platforms (Table 1). Probe element size and concentration. The element or spot size diameters range from 11 microns to ~200 microns in the different platforms. The size of the array elements (spots), their size in µ2 , and concentration in the number of molecules per spot are given in Table 2. There is also a large difference in the number of probe molecules per spot, with estimates from several million to hundreds of millions of molecules. This can heavily influence the kinetics of hybridization, signal quantification, and signal intensities of the probes, and these important factors will vary from platform to platform. Probe number per array. The number of probe sets may vary from 30,000 to 54,000, but the total number of probes per array actually ranges from about 30,000 to greater than 500,000 (Table 2). Microarrays may contain one probe per gene or up to twenty probes per gene. This fact alone can make it difficult to directly compare the data from platforms with such a wide range of the number of probes per mRNA sequence. Proper probe annotation. This is an intense area of investigation.6–8 The sequence databases for expressed genes are still in a state of flux, such that probe sequences derived from older databases may be dramatically different from the latest version. It has been found that some probe sequences no longer exist in the database, or were not annotated properly and now have different IDs or names. Thus, platforms may have probe sequences that do not exist in the genome or have the incorrect designation, and this has been an important source of confusion in the analysis of array data. Target preparation. There is no standard way of isolating RNA for target labeling, although almost all microarray experimentalists follow the rule of analyzing the integrity of their RNA samples before beginning labeling steps. Many expression profiling experiments in the past were uninterpretable simply because of poor RNA quality. A common method to test RNA integrity is through the use of an Agilent 2100 Bioanalyzer, which provides an electrophoretic tracing and a RNA integrity number (RIN) for judging RNA quality.9 Target synthesis. Targets are commonly synthesized via cDNA reactions on total RNA or by in vitro synthesis of linearly amplified RNA using T7 RNA polymerase technologies.10 The cDNA targets are thought to faithfully represent the original concentrations of the mRNA in the sample, but linearly amplified RNA may contain some bias in the original mRNA ratios.11,12 However,

202

the differences do not seem great, and the extra amount of target obtained by the amplification step is of some advantage when only small amounts of starting material are available; this can allow the detection of mRNAs with lower expression than targets from non-amplified samples. Target labeling. For detection of hybrids, the targets are commonly labeled with fluorescent dyes such as Cy3, Cy5, Alexa fluors, or phycoerythrin, and chemiluminescence is being used in a more recent addition to the array field (Applied Biosystems, Inc.). As every fluor or analyte will have a different stability, quantum efficiency, wavelength for stimulation/emission, and so on, the acquisition of images and quantification of signals adds another layer of complexity to comparison of data across different platforms. Hybridization/washing protocols. Not surprisingly, there are major differences in this area, and every commercial platform has their own method. Hybridization/washing protocols commonly vary in buffer types, salt concentration, temperature, time, volumes, and equipment. These variables are highly significant factors in the raw data, and can have a major impact on cross-platform comparisons. Imaging of arrays. Microarrays are imaged by confocal or non-confocal laser scanners, or by instruments detecting light from chemiluminescence reactions. The emission from the laser scan or chemiluminescence is detected and quantified by photomultiplier tubes (PMTs) or chargecoupled devices (CCDs) in combination with various electronic devices. Laser power, PMT or CCD settings, pixel sizes, scan time, and so on are not standardized. Needless to say, a great amount of variability in data acquisition can occur at this step.13 Data analysis. Early on in the microarray decade, a comment was made that there were more reviews about the new technology than there were actual research publications. Without too much exaggeration, a similar statement can be made about methods to analyze microarray data, where a large proportion of microarray articles have been statistical or bioinformatic in nature. Every major platform provider has their own favorite algorithm for analysis of the array data, and often several are provided for the same platform.14–16 At this point, there is no consensus about what is the best way to analyze microarray data, and, in fact, it has been demonstrated that different methods of analysis can result in a very different outcomes from the same data set.17 From the above discussion, it is obvious why there can be a great deal of discordance in results obtained from different or even the same microarray platforms. One might ask then, “How can you compare array data across

Journal of Biomolecular Techniques, Volume 17, issue 3, july 2006

The End of the Microarray Tower of Babel

Ta b l e 2 Physical Attributes of Some Common Arrays Array Type Affymetrix NimbleGen

Spot Size

Spot Area

Molecules/Spot

11–20 µ

100–400 µ2

“Millions”

cDNA (many sources)

100–150 µ

>10,000

µ2

100,000,000 (108)

Long Oligos (Agilent/ Illumina/Operon/etc.)

100–150 µ

>10,000 µ2

10,000 million (1010 )

different platforms?” or more bluntly, “Why do micro­ arrays work at all?” To answer these queries/criticisms, one should note that every platform has been validated to some extent by the manufacturers, and their product would not be in the public domain if they did not have confidence the microarrays were performing to their advertised specifications (in most cases), and the same holds true for the “home brew” arrays. In spite of this, many studies have reported that the correlation between different platforms is poor.18,19 On the other hand, other groups have shown good correlation, 20–23 while several others report a middling score for these types of studies. 24–26 How does one reconcile these somewhat contradictory findings? One can explain away much of the discrepancy by eliminating basic, trivial explanations or errors from platform comparisons. First, the same batch of RNA must be used for testing throughout in crossplatform studies. Expression from the same cell line can vary greatly from day to day, and different strains may turn out to be not even the same cell line at all. It is obvious, but not always easy to do in practice, that the same genes must be analyzed when cross-platform comparisons are done. When erroneous or mis-annotated probes are removed from analysis, correlations between platforms improve dramatically.7 Because of the large number of data points, it is tempting to overanalyze results from array experiments. It should be understood that the data from microarrays are usually neither highly precise nor very accurate in the sense that each data point is only an approximation of the “true” value. But, this does not detract from the usefulness of microarray technologies, as the expression profiling of the entire gene complement on one chip or slide is a remarkable accomplishment. In microarray studies, one should not place too much emphasis on the absolute magnitude of change in gene expression; rather, one should determine the reliability and significance of measurements demonstrating that certain genes are up- or down-regulated. When this is done, comparisons of array data across platforms generally show reasonable concordance. 22

Journal of Biomolecular Techniques, Volume 17, issue 3, july 2006

How Can We Improve Reliability/ Concordance of Microarray Data?

Although many of the arrays produced today are technological marvels, it is apparent from the above discussion that much more can be done to improve the reliability of results from these reagents. In the past few years, international consortia and technical groups have been formed to address these problems, and several will be briefly described below. These groups were chosen for discussion here as they are in constant communication with each other and many participants are members of all four groups. It is hoped that this cross-fertilization of ideas, concepts, and actions will lead to a much more standardized form of microarray technology for the research community. Microarray Gene Expression Database Society. The Microarray Gene Expression Database Society (MGED) was an international grass roots movement founded in late 1999 by the major microarray movers of the time, including Affymetrix, Stanford University, and the European Bioinformatics Institute (see www.mged.org for more information). The main goal of MGED was and is to provide a consistent, common basis by which data could be shared among scientists in the microarray field and other highthroughput genomics and proteomics areas. There are several areas in which the group hopes to standardize data input and reporting in order to end some of the frustration in the analysis and replication of microarray experiments. There are the Minimum Information About a Microarray Experiment (MIAME) standards, 27 which must be part of a manuscript before submitting for publication. The information must include enough facts to ensure that the data can be easily interpreted and have enough details for others to independently verify the analysis. There is a MicroArray and Gene Expression group, concerned with establishing standards for data management; the Ontology Working Group, concerned with accurately defining terms used to describe an experiment; and a Data Transformation and Normalization Group, focused on issues of data manipulation and quality assessment. The MGED

203

E.S. Kawasaki

ideas and suggestions have been adopted by several major journals and are a good step in taming and organizing the flood of unruly microarray data. National Institutes of Standards and Technolog y metrolog y. The National Institutes of Standards and Technology (NIST) has a group dedicated to studying the metrics of microarrays and gene expression profiling in general. 28 The gene expression metrology covers areas such as performance of microarrays, which might include differences in sensitivity, background, variability in signals, etc. They are also testing the majority of scanning instruments in common use and looking into methods for their validation and calibration. Other areas include RNA chemistry/ labeling, informatics, and statistical methods. The goal here is to provide the research community with solid facts about the basic metrics of microarray technologies, so that the researcher can make educated choices rather than guesses about experimentation in this complex area. MicroArray Quality Control. The MicroArray Quality Control (MAQC) project involves six FDA centers, commercial array and reagent providers, NIH/NCI, EPA, NIST, many academic laboratories, and other stakeholders. This large program, directed by the FDA, has involved over 30 centers and has generated data from over 1000 microarrays and quantitative PCR validation reactions. The purpose of the MAQC is to provide for the establishment of quality-control tools for the microarray community, in order to avoid and prevent common procedural failures in experiments. This will help researchers assess the performance of their arrays, and the project will also develop guidelines for data-analysis methods using the large reference data sets mentioned above. In addition, the same reference RNAs used in this study will be readily available to the public. Since all major array platforms were involved in this study, the microarray community and the FDA now have a large data set to objectively assess the technology for its relevance in research and the clinic.29,30 External RNA Controls Consortium: The External RNA Controls Consortium (ERCC) had its beginning in 2003 at a workshop held at Stanford University (Metrology and Standards Needs for Gene Expression Technologies: Universal RNA Standards), sponsored by NIST, Genomic Health Inc., Agilent Inc., and Affymetrix, Inc.31 The participants formed the ERCC in May of 2003 with the expressed goal of developing a set of external RNA standards designed as controls for use in all microarray platforms and quantitative PCR assays. There is a large group of scientists from over 50 international biotechnology, pharmaceutical, government, academic, and clinical organizations participating in this endeavor.32 As of this

204

writing, the consortium is close to delivering approximately 100 well-characterized clones that are to be used as controls for mammalian expression profiling and other systems as well. The controls are derived from random unique sequences as well as several bacterial and viral sequences. The standards are envisioned to be used for negative controls, ratiometric controls, dilution intensity controls, normalization, and so on. The clones as a reference set will be accessible through a public repository, and all the sequences and validation tests for these clones will be publicly available. The protocols for the preparation and use of the controls will be published, and algorithms and bioinformatic tools will be freely provided so that users can assess the utility of the data from use of the controls.33 This undertaking is of major significance to the microarray community because the members of the consortium have agreed to the condition that all microarray platforms and quantitative PCR systems will be using or have access to the same controls. If the ERCC is successful in this ambitious plan, it will not only be a first in the research community, but henceforth will provide a foundation by which expression profiling studies on all platforms can be more reliably compared through the presence of a common set of standards. The Tower of Harmony: Discord Turning Into Concord

We are now into the second decade of the microarray era. In the beginning, there were only a few genes available for study, but now chips containing entire genomes are commonplace. The major complaint about this technology has been that it is unreliable and microarray experiments can demonstrate any outcome the investigator wishes to prove.34–38 Ten years is not a long period for a technology to mature; PCR is now two decades old and still produces research titles in journals such as: “Relative transcript quantification by quantitative PCR: Roughly right or precisely wrong?”39 There are many reasons why optimism is warranted for the future of the microarray sciences.40–42 The DNA sequences of most of the expressed genes are now well documented, so that design and manufacture of probes is much more accurate and robust than before. The quality of commercial arrays of all types has improved greatly, and the home-brew chips are also quite satisfactory.43 The causes of confusion in the past are better understood, and the obvious errors in the technology can be avoided. A consistent and standardized way of reporting results (MGED) will greatly facilitate the replication of experiments and comparison of data. The work on the metrology of microarray technology (NIST) will provide researchers with the knowledge to judge the nuts and

Journal of Biomolecular Techniques, Volume 17, issue 3, july 2006

The End of the Microarray Tower of Babel

bolts of their experiments with a critical eye, which should result in more reliable data in the end. The recommendation of quality control tools for microarrays (MAQC) is an excellent way for the microarray community to judge the performance of their arrays, which will in turn make it easier to accept data from many different types of platforms. Finally, the implementation of universal standards (ERCC) will provide an accepted set of controls for every array platform, and allow cross-comparison and correlation of all related expression experiments. In short, the future for microarrays looks bright, and concordance in data from different platforms will soon be the rule rather than the exception. References 1. Schena M, Shalon D, Davis RW, Brown PO. Quantitative monitoring of gene expression patterns with a complementary DNA microarray. Science 1995;270:368–371. 2. Lockhart DJ, Dong H, Byrne MC, Follettie MT, Gallo MV, Chee MS, et al. Expression monitoring by hybridization to high-density oligonucleotide arrays. Nat Biotechnol 1996;14:1675–1680. 3. Barrett JC, Kawasaki ES. Microarrays: The use of oligonucleotides and cDNA for the analysis of gene expression. Drug Discovery Today 2003;8:134–141. 4. Ramdas L, Cogdell DE, Jia JY, Taylor EE, Dunmire VR, Hu L, et al. Improving signal intensities for genes with low-expression on oligonucleotide arrays. BMC Genomics 2004;5:35. ������������ 5. Debey S, Zander ��������������������������������������������������� T, Brors B, Popov A, Eils R, Schultze JL. �� A highly standardized, robust, and cost-effective method for genome-wide transcriptome analysis of peripheral blood applicable to large-scale clinical trials. Genomics 2006;87(5):653–664. 6. Jones W. Microarray probes that mask differential expression. Pharmaceutical Discovery, 2005;June:16–20. 7. Carter SL, Eklund AC, Mecham BH, Kohane IS, Szallasi Z. Redefinition of Affymetrix probe sets by sequence overlap with cDNA microarray probes reduces cross-platform inconsistencies in cancer-associated gene expression measurements. BMC Bioinformatics 2005;6:107. ��������������������������������������������������������������� 8. Dai M, Wang P, Boyd AD, Kostov G, Athey B, Jones EG, et al. Evolving gene/transcript definitions significantly alter the interpretation of GeneChip data. Nucleic Acids Res 2005;33:e175. 9. Schroeder A, Mueller O, Stocker S, Salowsky R, Leiber M, Gassmann M, et al. The RIN: An RNA integrity number for assigning integrity values to RNA measurements. BMC Mol Biol 2006;7:3. 10. Zhu B, Xu F, Baba Y. An evaluation of linear RNA amplification in cDNA microarray gene expression analysis. Molec Genet & Metab 2006;87:71–79. 11. Nygaard V, Hovig E. Options available for profiling small samples: A review of sample amplification technology when combined with microarray profiling. Nucleic Acids Res 2006;34:996– 1014. 12. Cope L, Hartman SM, Gohlmann HWH, Tiesmann JP, Irizarry RA. Analysis of Affymetrix GeneChip® data using amplified RNA. Biotechniques 2006;40:165–170. ���������������������� 13. Shi L, Tong W, Su ����������������������������������������� Z, Han T, Han J, Puri RK, et al. Microar�������� ray scanner calibration curves: Characteristics and implications. BMC Bioinformatics 2005;6(Suppl 2):S11. 14. Ploner A, Miller LD, Hall P, Bergh J, Pawitan Y. Correlation test to assess low-level processing of high-density oligonucleotide microarray data. BMC Bioinformatics 2005;6:80.

Journal of Biomolecular Techniques, Volume 17, issue 3, july 2006

�������������������������������������������������������������� 15. Shedden K, Chen W, Kuick R, Ghosh D, Macdonald J, Cho KR, et al. ����������������������������������������������������� Comparison of seven methods for producing Affymetrix expression scores based on False Discovery Rates in disease profiling data. BMC Bioinformatics 2005;6:26. 16. Garge NR, Page GP, Sprague AP, Gorman BS, Allison DB. Reproducible clusters from microarray research: Whither? BMC Bioinformatics 2005;6(Suppl 2):S10. 17. Harr B, Schlotterer C. Comparison of algorithms for the analysis of Affymetrix microarray data as evaluated by co-expression of genes in known operons. Nucleic Acids Res 2006;34:e8. 18. Tan PK, Downey TJ, Spitznagel EL, Xu P, Fu D, Dimitrov DS, et al. Evaluation of gene expression measurements from commercial microarray platforms. Nucleic Acids Res 2003;31:5676–5684. 19. Shi L, Tong W, Fang H, Scherf U, Han J, Puri RK, et al. Crossplatform comparability of microarray technology: Intra-platform consistency and appropriate data analysis procedures are essential. BMC Bioinformatics 2005;6(Suppl 2):S12. 20. Yauk CL, Berndt ML, Williams A, Douglas GR. Comprehensive comparison of six microarray technologies. Nucleic Acids Res 2004;32:e124. 21. Woo Y, Affourtit J, Daigle S, Viale A, Johnson K, Naggert J, et al. A comparison of cDNA, oligonucleotide, and Affymetrix GeneChip gene expression microarray platforms. J Biomolec Techniques 2004;15:276–284. ��������������������������������������������������������������� 22. Petersen D, Chandramouli GV, Geoghegan J, Hilburn J, Paarlberg J, Kim CH, et al. Three ���������������������������������������� microarray platforms: An analysis of their concordance in profiling gene expression. BMC Genomics 2005;6:63. 23. Warnat P, Eils R, Brors B. Cross-platform analysis of cancer microarray data improves gene expression based classification of phenotypes. BMC Bioinformatics 2005;6:265. 24. Larkin JE, Frank BC, Gavras H, Sultana R, Quackenbush J. Independence and reproducibility across microarray platforms. Nat Methods 2005;2:337–343. 25. Irizarry RA, Warren D, Spencer F, Kim IF, Biswal S, Frank BC, et al. Multiple-laboratory comparison of microarray platforms. Nat Methods 2005;2:345–349. 26. Members of the Toxicogenomics Research Consortium. Standardizing global gene expression analysis between laboratories and across platforms. Nat Methods 2005;2:351–356. 27. Brazma A, Hingamp P, Quackenbush J, Sherlock G, Spellman P, Stoeckert C, et al. Minimum information about a microarray experiment (MIAME)—towards standards for microarray data. Nat Genetics 2001;29:365–371. 28. For more information see www.nist.gov and contact Dr. Marc Salit at [email protected]. 29. For more information on MAQC, see website: www.fda.gov/ nctr/science/centers/toxicoinformatics/maqc/index.htm 30. Tezak Z, Ranamukhaarachchi D, Russek-Cohen E, Gutman SI. FDA perspectives on potential microarray-based clinical diagnostics. Human Genomics 2006;2:236–243. 31. Cronin M, Ghosh K, Sistare F, Quackenbush J, Vilker V, O’Connell C. Universal RNA reference materials for gene expression. Clin Chem 2004;50:1464–1471. 32. The External RNA Controls Consortium. The External RNA Controls Consortium: A progress report. Nat Methods 2005;2:731–734. 33. External RNA Controls Consortium. Proposed methods for testing and selecting the ERCC external RNA controls. BMC Genomics 2005;6:150. 34. Ioannidis JPA. Microarrays and molecular research: Noise discovery? Lancet 2005;365:454–455. 35. Michiels S, Koscielny S, Hill C. Prediction of cancer outcome with microarrays: A multiple random validation strategy. Lancet 2005;365:488–492. 36. Shields R. MIAME we have a problem. Trends Genetics 2006;22:65– 66.

205

E.S. Kawasaki

37. Miron M, Nadon R. Inferential literacy for experimental highthroughput biology. Trends Genetics 2006;22:84–89. 38. Draghici S, Khatri P, Eklund AC, Szallasi Z. Reliability and reproducibility issues in DNA microarray measurements. Trends Genetics 2006;22:101–109. 39. Skern R, Frost P, Nilsen F. Relative transcript quantification by quantitative PCR: Roughly right or precisely wrong? BMC Molec Biol 2005;6:10.

206

40. van Bakel H, Holstege FCP. In control: Systematic assessment of microarray performance. EMBO reports 2004;5:964–969. 41. Allison, DB, Cui X, Page GP, Sabripour M. Microarray data analysis: From disarray to consolidation and consensus. Nat Rev Genetics 2005;7:55–65. 42. Hoheisel JD. Microarray technology: Behond transcript profiling and genotype analysis. Nat Rev Genetics 2006;7:200–210. 43. Elvidge G. Microarray expression technology: From start to finish. Pharmacogenomics 2006;7:123–134.

Journal of Biomolecular Techniques, Volume 17, issue 3, july 2006