This documentation is for CPAT v3.0.0 or future versions. For documentation of CPAT v2.0.0 and older versions, please go to http://rna-cpat.sourceforge.net/
CPAT v3.0.4 (05/26/2021)
Fix bug to read remote file for Python3.
CPAT v3.0.3 (03/08/2021)
Update “cpat.py” to handle alternative start codens.
CPAT v3.0.2 (08/17/2020)
Update “make_logitModel.py” to make it compatible with “cpat.py”.
Minor bug fixed regarding the output format.
For many transcripts, the longest ORF may not be the real ORF. For example, in human genome, the 2nd longest ORF of NM_198086 is the real ORF, and the 3rd longest ORF of NM_030915 is the real ORF. Version 3.0.0 is released to address this problem.
- If model is provided, CPAT can be used as an ORFfinder. It gives exactly the same results as NCBI ORFfinder does.
- Search for all ORF candidates. The number of ORF reported is controlled by
- In addition to basic ORF information (“ORF frame”, “ORF strand”, “ORF start”, “ORF end”, “ORF sequence”), it also reports “coding probability” for each ORF.
- The best ORF will be selected (controlled by
--best-orf) either by ORF length or coding probability.
CPAT is a bioinformatics tool to predict RNA’s coding probability based on the RNA sequence characteristics. To achieve this goal, CPAT calculates scores of these 4 linguistic features from a set of known protein-coding genes and another set of non-coding genes.
CPAT will then builds a logistic regression model using these 4 features as predictor variables and the “protein-coding status” as the response variable. After evaluating the performance and determining the probability cutoff, the model can be used to predict new RNA sequences.
Run CPAT online¶
https://wlcb.oit.uci.edu/cpat is hosted by Dr Wei Li’s lab @ University of California Irvine.
Step1: Upload data to CPAT server. There are 3 different ways to uploada
- Upload BED or FASTA format files from local disk. Files can be regular or compressed (*.gz, *.Z. *.z, *.bz, *.bz2, *.bzip2).
- For small dataset, user can copy and paste data (in BED or FASTA format) directly to the text area.
- For extremely larger dataset, user can save data in web server (http, https or ftp) first, then paste the data url to text area.
Step2: Select Select Species assembly
Step3: Click Submit button
- This web server only supports Human (hg19), Mouse (mm9 and mm10), Fly (dm3) and Zebrafish (Zv9).
- When input file is BED format, the reference genome is required and the assembly version is important.
- When input file is FASTA format, the reference genome and the assembly version is ignored.
Run CPAT on local computer¶
User need to provide a gene file (‘-g’), a logit model file (‘-d’), a hexamer frequency table file (‘-x’) and specify the output file name(‘-o’). Gene file could be either in BED. or FASTA format. If in BED format, user also needs to specify reference genome sequence (‘-r’).
- BED format file (regular text or compressed). BED file should be in standard 12-column format.
- FASTA format file (regular text or compressed)
- a URL pointing to data that are saved remotely (data could be either BED or FASTA, either regular text or compressed file). http://, https:// and ftp:// are supported. A remote file cannot be compressed.
Command line options¶
--version show program’s version number and exit -h, --help show this help message and exit -g GENE_FILE, --gene=GENE_FILE Genomic sequnence(s) of RNA in FASTA (https://en.wikipedia.org/wiki/FASTA_format) or standard 12-column BED (https://genome.ucsc.edu/FAQ/FAQformat.html#format1) format. It is recommended to use short and unique sequence identifiers (such as Ensembl transcript id) in FASTA and BED file. If this is a BED file, reference genome (‘-r/–ref’) should be specified. The input FASTA or BED file could be a regular text file or compressed file (*.gz, *.bz2) or accessible URL (http://, https://, ftp://). -o OUT_FILE, --outfile=OUT_FILE The prefix of output files. -d LOGIT_MODEL, --logitModel=LOGIT_MODEL Logistic regression model. The prebuilt models for Human, Mouse, Fly, Zebrafish are availablel. Run ‘make_logitModel.py’ to build logistic regression model out of your own training datset. -x HEXAMER_DAT, --hex=HEXAMER_DAT The hexamer frequency table. The prebuilt tables for Human, Mouse, Fly, Zebrafish are availablel. Run ‘make_hexamer_tab.py’ to make this table out of your own training dataset. -r REF_GENOME, --ref=REF_GENOME Reference genome sequences in FASTA format. Reference genome file will be indexed automatically (produce *.fai file along with the original *.fa file within the same directory) if hasn’t been done. Ignore this option if FASTA file was provided to ‘-g/–gene’. --antisense Also search for ORFs from the anti-sense strand. Sense strand (or coding strand) is DNA strand that carries the translatable code in the 5′ to 3′ direction. default=False (i.e. only search for ORFs from the sense strand) --start=START_CODONS Start codon used by ORFs. Use ‘T’ instead of ‘U’. default=ATG --stop=STOP_CODONS Stop codons used by ORFs. Multiple stop codons should be separated by ‘,’. Use ‘T’ instead of ‘U’. default=TAG,TAA,TGA --min-orf=MIN_ORF_LEN Minimum ORF length in nucleotides. default=75 --top-orf=N_TOP_ORF Number of ORF candidates. Many RNAs have dozens of possible ORFs, in most cases, the real ORF is ranked (by size) in the top several. To increase speed, we do not need to calculate “Fickett score”, “Hexamer score” and “coding probability” for all of them. default=5 --width=LINE_WIDTH Line width of output ORFs in FASTA format. default=100 --log-file=LOG_FILE Name of log file. default=”CPAT_run_info.log” --best-orf=MODE Criteria to select the best ORF: “l”=length, selection according to the “ORF length”; “p”=probability, selection according to the “coding probability”. default=”p” --verbose Logical to determine if detailed running information is printed to screen.
Use FASTA file as input:
$ cpat.py -x Human_Hexamer.tsv --antisense -d Human_logitModel.RData --top-orf=5 -g Human_test_coding_mRNA.fa -o output1
Use remote FASTA file as input:
$ cpat.py -x Human_Hexamer.tsv --antisense -d Human_logitModel.RData --top-orf=5 -g https://data.cyverse.org/dav-anon/iplant/home/liguow/CPAT/Human_test_coding_mRNA.fa -o output2
Use BED file as input. ‘-r’ is required:
$ cpat.py -x Human_Hexamer.tsv --antisense -d Human_logitModel.RData --top-orf=5 -g Human_test_coding_mRNA_hg19.bed -r hg19.fa -o output3
- output.ORF_seqs.fa: The top ORF sequences (at least 75 nucleotides long) in FASTA format.
- output.ORF_prob.tsv: ORF information (strand, frame, start, end, size, Fickett TESTCODE score, Hexamer score) and coding probability)
- output.ORF_prob.best.tsv: The information of the best ORF. This file is a subset of “output.ORF_prob.tsv”
- output.no_ORF.txt: Sequence IDs or BED entries with no ORF found. Should be considered as non-coding.
- output.r: Rscript file.
- CPAT_run_info.log: log file
Build hexamer table¶
make_hexamer_tab.py calculates the in frame hexamer (6mer) frequency from CDS sequence in fasta format. The CDS is mRNA sequence that removes UTR. This table is required by CPAT to calculate the hexamer usage score. Users can download prebuilt hexamer tables (Human, Mouse, Fly, Zebrafish) from here
--version show program’s version number and exit -h, --help show this help message and exit -c CODING_FILE, --cod=CODING_FILE Coding sequence (must be CDS without UTR, i.e. from start coden to stop coden) in fasta format. User can get CDS sequence of a bed file using UCSC table browser -n NONCODING_FILE, --noncod=NONCODING_FILE Noncoding sequences in fasta format
$ make_hexamer_tab.py -c Human_coding_transcripts_CDS.fa -n Human_noncoding_transcripts_RNA.fa >Human_Hexamer.tsv $ head Human_Hexamer.tsv hexamer coding noncoding GAACGT 0.000114999540425 6.20287252729e-05 CTTCTT 0.000280298143192 0.000464526231488 CACCCT 0.000254883880114 0.000337895737524 GAACGG 0.000178535198119 5.8077265737e-05 GAACGC 0.000136389878516 6.03746259323e-05 GAACGA 0.00015830968042 5.87205265917e-05 CACCCA 0.000258696019576 0.000448628498937 CTTCTA 0.000147508618612 0.000280645521457 CACCCC 0.000328479350276 0.000342582352322 ...
Build Logit model¶
Build logistic regression model (“prefix.logit.RData”) required by CPAT. This program will output 3 files:
- prefix.feature.xls: A table contains features calculated from training datasets (coding and noncoding gene lists).
- prefix.logit.RData: logit model required by CPAT (if R was installed).
- prefix.make_logitModel.r: R script to build the above logit model.
Note: Users can download prebuilt logit models (Human, Mouse, Fly, Zebrafish) from here
--version show program’s version number and exit -h, --help show this help message and exit -c CODING_FILE, --cgene=CODING_FILE Genomic sequnences of protein-coding RNAs in FASTA (https://en.wikipedia.org/wiki/FASTA_format) or standard 12-column BED (https://genome.ucsc.edu/FAQ/FAQformat.html#format1) format. It is recommended to use short and unique sequence identifiers (such as Ensembl transcript id) in FASTA and BED file. The input FASTA or BED file could be a regular text file or compressed file (*.gz, *.bz2) or accessible URL (http://, https://, ftp://). When BED file is provided, use the ORF defined in the BED file (the 7th and 8th columns in BED file define the positions of ‘start codon, and ‘stop codon’, respectively). When FASTA file is provided, searching for the longet ORF. For well annotated genome, we recommend using BED file as input because the longest ORF predicted from RNA sequence might not be the real ORF. If this is a BED file, reference genome (‘-r/–ref’) should be specified. -n NONCODING_FILE, --ngene=NONCODING_FILE Genomic sequences of non-coding RNAs in FASTA (https://en.wikipedia.org/wiki/FASTA_format) or standard 12-column BED (https://genome.ucsc.edu/FAQ/FAQformat.html#format1) format. It is recommended to use short and unique sequence identifiers (such as Ensembl transcript id) in FASTA and BED file. The input FASTA or BED file could be a regular text file or compressed file (*.gz, *.bz2) or accessible URL (http://, https://, ftp://). If this is a BED file, reference genome (‘-r/–ref’) should be specified. -o OUT_FILE, --outfile=OUT_FILE The prefix of output files. -x HEXAMER_DAT, --hex=HEXAMER_DAT Hexamer frequency table. CPAT has prebuilt hexamer frequency tables for Human, Mouse, Fly, Zebrafish. Run ‘make_hexamer_tab.py’ to generate this table. -r REF_GENOME, --ref=REF_GENOME Reference genome sequences in FASTA format. Ignore this option if mRNA sequences file was provided to ‘-g’. Reference genome file will be indexed automatically if the index file *.fai) does not exist. -s START_CODONS, --start=START_CODONS Start codon (use ‘T’ instead of ‘U’) used to define the start of open reading frame (ORF). default=ATG -t STOP_CODONS, --stop=STOP_CODONS Stop codon (use ‘T’ instead of ‘U’) used to define the end of open reading frame (ORF). Multiple stop codons are separated by ‘,’. default=TAG,TAA,TGA --min-orf=MIN_ORF_LEN Minimum ORF length in nucleotides. default=30 --log-file=LOG_FILE Name of log file. default=”make_logitModel_run_info.log” --verbose Logical to determine if detailed running information is printed to screen.
$ make_logitModel.py -x Human_Hexamer.tsv -c Human_coding_transcripts_mRNA.fa -n Human_noncoding_transcripts_RNA.fa -o Human Process protein coding transcripts: Human_coding_transcripts_mRNA.fa Input gene file is in FASTA format Process non coding transcripts: Human_noncoding_transcripts_RNA.fa Input gene file is in FASTA format build logi model ... Warning message: glm.fit: fitted probabilities numerically 0 or 1 occurred #or use BED file as input $ make_logitModel.py -x Human_Hexamer.tsv -c Human_coding_transcripts_hg19.bed -n Human_noncoding_transcripts_hg19.bed -r /database/hg19.fa -o Human
Use CPAT to detect ORF¶
It is perfectly fine to use CPAT to find ORFs. And CPAT will gives exactly the same results as NCBI ORFfinder
Below is the mRNA sequence of protein-coding gene UQCR10. Copy and save it as “test.fa”.
>NM_013387.4 GCGGTGGCGCGAGTTGGACTGTGAAGAAACATGGCGGCCGCGACGTTGAC TTCGAAATTGTACTCCCTGCTGTTCCGCAGGACCTCCACCTTCGCCCTCA CCATCATCGTGGGCGTCATGTTCTTCGAGCGCGCCTTCGATCAAGGCGCG GACGCTATCTACGACCACATCAACGAGGGGAAGCTGTGGAAACACATCAA GCACAAGTATGAGAACAAGTAGTTCCTTGGAGGCCCCCATCCAGGCCAGA AGGACCAGGTCCACCCAGCAGCTGTTTGCCCAGAGCTGGAGCCTCAGCTT GAAGATGATGCTCAAGGTACTCTTCATGGACCACCATTCGCTGTTGGCAA GAAACGGCTTTACTTACAAAACAGACTCTTTACCTTCTGCTGTGTTTGAA GTATGTTTAGTCAGCATGCTCAGGAAATAAATGTGAATTGCCCTTGAGAC CTGCTTCTACATTGGTTGCTTTGTTAACTCTACCTGATCTTCACTTGTCA GTAATTTGAGACCACTTCAAAGCCCTCTGCAAACACCCCAAAGGCAGAAT CTGCTATTTTGAGTTTTCCATTAACTTCCAAAGAATTCTGGTTTTCAAAA CAGGAGCCAGAGTTGGAGATATTACAGTCAACTTTGGCTTCTAAGCCAGT AATTCCATTCTTAAATACCTCACTGTCTTGGCCATGGGGAAGCACTATGG CCTCAGCTGGGGGAAAGACCCTGGCCTAGGGGTCTTAGCCACTCCCCACC CTAGGGTATAGTTCAGGGGTATCCAATCCTTTGGCTTCCCTGGGCCATGT TGGAAGAATTGTCTTGGGCCACACATAAAATACAGTAACCATAGCTGATG AGCTAAAACAAAAAACAATGGTTTGTGCAAAAATCTCATAATGTTTTAAT AAAGTTGAAGAATTTG
The command to run cpat.py is as below
$ cpat.py -x Human_Hexamer.tsv -d Human_logitModel.RData --top-orf=100 --antisense -g test.fa -o output
- You must specify
--antisense, otherwise, it will only search ORFs from the sense strand.
- You also specify
--top-orfto a big number to report all the ORFs.
--min-orfis set to 75 by default, same as NCBI ORFfinder.
Check the results¶
A total of 8 ORFs were found (sorted by the ORF size, the 7th column). If you copy and paste the same sequence to NCBI ORFfinder web server, you will get exactly the same results.
$more output.ORF_prob.tsv ID mRNA ORF_strand ORF_frame ORF_start ORF_end ORF Fickett Hexamer Coding_prob NM_013387.4_ORF_1 916 - 2 327 1 327 1.103 0.28998918917275 0.792763525921043 NM_013387.4_ORF_2 916 + 2 209 430 222 1.1605 0.0674464550896935 0.271842476390681 NM_013387.4_ORF_3 916 - 1 889 695 195 0.9192 -0.32000518247443 0.0113140534730678 NM_013387.4_ORF_4 916 + 1 31 222 192 1.2952 0.600469985268255 0.915129459422605 NM_013387.4_ORF_5 916 - 1 337 197 141 1.1626 0.133867810597757 0.185245402415541 NM_013387.4_ORF_6 916 - 3 119 3 117 1.2673 0.442351820001225 0.618496534888714 NM_013387.4_ORF_7 916 - 3 842 735 108 0.5832 -0.19401829042094 0.00290794398512764 NM_013387.4_ORF_8 916 + 3 684 761 78 0.7415 -0.154613060436537 0.00454929869181486
CPAT also provides Fickett’s TESTCODE score, Hexamer score and coding probability for each ORF, to help you determine which one is more likely the real ORF. For most mRNAs, the largest ORF is also the most likely one, but not always. In this particular example, ORF_4 is the most likely one to code for protein since it has the highest coding probability (plese note, ORF_4 is not the largest ORF of NM_013387.4). This can be demonstrated by BLATing the 8 ORF sequences to the reference genome.
How to choose cutoff¶
Optimum cutoff were determined from TG-ROC.
- Human coding probability (CP) cutoff: 0.364 (CP >=0.364 indicates coding sequence, CP < 0.364 indicates noncoding sequence) (see performance figure D)
- Mouse coding probability (CP) cutoff: 0.44
- Fly coding probability (CP) cutoff: 0.39
- Zebrafish coding probability (CP) cutoff: 0.38
1) Download R code and data from here
2) Put the R code and the data table in the same folder
$ ls 10Fold_CrossValidation.r Human_train.dat
3) Run the R code from command line or console. The R code will perform 10-fold cross validation and generate Figure_3.
$ Rscript 10Fold_CrossValidation.r # install ROCR before running this code Loading required package: gplots Attaching package: ‘gplots’ The following object is masked from ‘package:stats’: lowess Loading required package: methods Warning message: package ‘gplots’ was built under R version 3.1.2  "ID" "mRNA" "ORF" "Fickett" "Hexamer" "Label" Warning message: glm.fit: fitted probabilities numerically 0 or 1 occurred Warning message: glm.fit: fitted probabilities numerically 0 or 1 occurred Warning message: glm.fit: fitted probabilities numerically 0 or 1 occurred Warning message: glm.fit: fitted probabilities numerically 0 or 1 occurred Warning message: glm.fit: fitted probabilities numerically 0 or 1 occurred Warning message: glm.fit: fitted probabilities numerically 0 or 1 occurred Warning message: glm.fit: fitted probabilities numerically 0 or 1 occurred Warning message: glm.fit: fitted probabilities numerically 0 or 1 occurred Warning message: glm.fit: fitted probabilities numerically 0 or 1 occurred Warning message: glm.fit: fitted probabilities numerically 0 or 1 occurred null device 1
How to prepare training dataset¶
We prebuild hexamer tables and logit models for human, mouse, fly and zebrafish. If you want to run CPAT for other species, you need to prepare your own training data. These two files are required when you run make_hexamer_tab.py and make_logitModel.py.
- It’s better to have balanced training dataset (i.e. the number of coding sequences is roughly equal to the number of noncoding sequences).
- If the genome of the species you are working on is NOT well annotated and does not have enough “coding” and “noncoding” genes to build the training data, you could build your model using data from other species that is evolutionary close to the species you are working on.
Combinatorial effects of 3 major features. 10,000 coding genes (red dots) and 10,000 noncoding genes (blue dots) are clearly separated into two clusters. (below figure)
Performance evaluation using 10-fold cross validation (10,000 coding genes and 10,000 noncoding genes). Blue dotted curves represent the 10-fold cross validations, red solid curve represents the averaged curve between 10 runs of validations. (A) ROC curve. (B) PR (precision-recall) curve. (C) Accuracy vs cutoff value. (D) Two graphic ROC curve to determine the optimum cutoff value.
To compare CPAT with CPC and PhyloCSF, we build an independent testing dataset that composed of 4,000 high quality protein coding genes from Refseq annotation and 4,000 lincRNAs from Human lincRNA catalog (Cabili et al., 2011). All 8000 genes were not included in the training dataset of CPAT.
CPAT is distributed under GNU General Public License
This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
Wang, L., Park, H. J., Dasari, S., Wang, S., Kocher, J.-P., & Li, W. (2013). CPAT: Coding-Potential Assessment Tool using an alignment-free logistic regression model. Nucleic Acids Research, 41(6), e74. doi:10.1093/nar/gkt006
- Liguo Wang: wang.liguo AT mayo.edu
- Wei Li: wei.li AT uci.edu