EVA home   EVA e-mail   EVA mirrors   -   Secondary structure   Comparative modelling   Threading   Contacts
 

Statistical analysis of Eva-CM results.

The Student t distribution statistics are used to assess the statistical significance of ranking of protein structure modeling methods (details).

DISCLAIMER: Any final ranking list will be frustrated in the sense that method i assessed to be better than method j in the direct pairwise comparison may be positioned worse than method j in the final ranking because of the impact of the pairwise comparisons with other methods, based on different sets of common models.


Analysis of Fold accuracy (% Equivalent Positions):
    Ranking of the methods:

      1. sdsc1
      2. 3djigsaw
      3. SwissModel
      4. cphmodels
      5. esypred

    t-Student statistical analysis of the comparisons:
    sdsc13djigsawSwissModelcphmodelsesypred
    1. sdsc1 1.42 ± 2.02
    [ 10] ( 2)    		 1.73 ± 16.72
    [ 1866] ( 1866)    		 2.24 ± 13.66
    [ 374] ( 374)    		< 2 models
    2. 3djigsaw -1.42 ± 2.02
    [] ( 2)    		 0.17 ± 11.47
    [] ( 4450)    		 -0.80 ± 14.48
    [] ( 114)    		 1.33 ± 11.91
    [ 5824] ( 5824)
    3. SwissModel -1.73 ± 16.72
    [ 1866] ( 1866)    		 -0.17 ± 11.47
    [] ( 4450)    		 1.86 ± 15.81
    [ 1082] ( 1082)    		 0.87 ± 12.19
    [ 4907] ( 4907)
    4. cphmodels -2.24 ± 13.66
    [ 374] ( 374)    		 0.80 ± 14.48
    [] ( 114)    		 -1.86 ± 15.81
    [ 1082] ( 1082)    		 -1.61 ± 13.55
    [ 373] ( 373)
    5. esypred< 2 models -1.33 ± 11.91
    [ 5824] ( 5824)    		 -0.87 ± 12.19
    [ 4907] ( 4907)    		 1.61 ± 13.55
    [ 373] ( 373)

Analysis of Alignment accuracy (% of correct aligned positions):
    Ranking of the methods:

      1. SwissModel
      2. sdsc1
      3. 3djigsaw
      4. cphmodels
      5. esypred

    t-Student statistical analysis of the comparisons:
    SwissModelsdsc13djigsawcphmodelsesypred
    1. SwissModel 2.79 ± 20.67
    [ 1866] ( 1866)    		 0.89 ± 19.20
    [ 4450] ( 4450)    		 4.03 ± 25.00
    [ 1082] ( 1082)    		 4.68 ± 25.23
    [ 4907] ( 4907)
    2. sdsc1 -2.79 ± 20.67
    [ 1866] ( 1866)    		 1.67 ± 2.35
    [ 10] ( 2)    		 0.08 ± 17.19
    [] ( 374)    		< 2 models
    3. 3djigsaw -0.89 ± 19.20
    [ 4450] ( 4450)    		 -1.67 ± 2.35
    [] ( 2)    		 2.57 ± 27.86
    [] ( 114)    		 4.07 ± 25.59
    [ 5824] ( 5824)
    4. cphmodels -4.03 ± 25.00
    [ 1082] ( 1082)    		 -0.08 ± 17.19
    [] ( 374)    		 -2.57 ± 27.86
    [] ( 114)    		 -2.50 ± 28.18
    [] ( 373)
    5. esypred -4.68 ± 25.23
    [ 4907] ( 4907)    		< 2 models -4.07 ± 25.59
    [ 5824] ( 5824)    		 2.50 ± 28.18
    [] ( 373)

Analysis of RMSD (Ca global rmsd):
    Ranking of the methods:

      1. SwissModel
      2. sdsc1
      3. 3djigsaw
      4. cphmodels
      5. esypred

    t-Student statistical analysis of the comparisons:
    SwissModelsdsc13djigsawcphmodelsesypred
    1. SwissModel -0.32 ± 2.05
    [ 1866] ( 1866)    		 0.03 ± 2.10
    [] ( 4450)    		 -0.23 ± 2.10
    [ 1082] ( 1082)    		 -0.23 ± 2.25
    [ 4907] ( 4907)
    2. sdsc1 0.32 ± 2.05
    [ 1866] ( 1866)    		 -0.35 ± 0.49
    [] ( 2)    		 -0.02 ± 1.02
    [] ( 374)    		< 2 models
    3. 3djigsaw -0.03 ± 2.10
    [] ( 4450)    		 0.35 ± 0.49
    [ 10] ( 2)    		 -0.11 ± 1.82
    [] ( 114)    		 -0.27 ± 2.28
    [ 5824] ( 5824)
    4. cphmodels 0.23 ± 2.10
    [ 1082] ( 1082)    		 0.02 ± 1.02
    [] ( 374)    		 0.11 ± 1.82
    [] ( 114)    		 0.23 ± 2.22
    [ 373] ( 373)
    5. esypred 0.23 ± 2.25
    [ 4907] ( 4907)    		< 2 models 0.27 ± 2.28
    [ 5824] ( 5824)    		 -0.23 ± 2.22
    [ 373] ( 373)

Analysis of % of modelled residues (% of Coverage):
    Ranking of the methods:

      1. sdsc1
      2. SwissModel
      3. 3djigsaw
      4. cphmodels
      5. esypred

    t-Student statistical analysis of the comparisons:
    sdsc1SwissModel3djigsawcphmodelsesypred
    1. sdsc1 3.67 ± 16.05
    [ 1866] ( 1866)    		 0.00 ± 0.00
    [] ( 2)    		 2.68 ± 12.92
    [ 374] ( 374)    		< 2 models
    2. SwissModel -3.67 ± 16.05
    [ 1866] ( 1866)    		 -1.28 ± 11.18
    [ 4450] ( 4450)    		 0.66 ± 13.59
    [] ( 1082)    		 -1.32 ± 10.43
    [ 4907] ( 4907)
    3. 3djigsaw 0.00 ± 0.00
    [] ( 2)    		 1.28 ± 11.18
    [ 4450] ( 4450)    		 -0.82 ± 5.64
    [] ( 114)    		 0.02 ± 7.34
    [] ( 5824)
    4. cphmodels -2.68 ± 12.92
    [ 374] ( 374)    		 -0.66 ± 13.59
    [] ( 1082)    		 0.82 ± 5.64
    [] ( 114)    		 -0.66 ± 8.95
    [] ( 373)
    5. esypred< 2 models 1.32 ± 10.43
    [ 4907] ( 4907)    		 -0.02 ± 7.34
    [] ( 5824)    		 0.66 ± 8.95
    [] ( 373)
 

  EVA home   EVA e-mail   EVA mirrors   -   Secondary structure   Comparative modelling   Threading   Contacts