Autarchy of the Private Cave

• Categories

  Select Category*nixAffiliate programsArtificial Intelligence   Agents   Machine learning   Robotics   TechnologiesBooksComparisonHardwarehow-toHumourKyivLifeLinksMiscMoviesNotepadPersonalProgramming   Ada   OS   PHP   Python   XHTML/CSSqotdRantScience   Bioinformatics   Systems BiologySocietySoftware   SecurityStartupsUkraineWeb   CMS   Drupal   WP PlugInsWelfare

• Tags list

  blog btrfs bug CMS convert data database Debian development drupal example ffmpeg FireFox fix free gallery2 git google HDD hosting how-to http linux microarray module mysql package performance PHP plugin problem project management Python R repository server shared hosting spam statistics tool Ukraine update upgrade windows wordpress

• Related entries

  No related content found.

• Subscribe

  

  Subscribe to RSS feed.

  Desktop Reader Bloglines Feedly Live Netvibes Yahoo!

• Archives

  Select Month December 2022 October 2022 October 2018 August 2018 July 2018 September 2017 August 2017 February 2017 December 2016 June 2016 May 2016 April 2016 March 2016 February 2016 January 2016 December 2015 July 2015 May 2015 April 2015 March 2015 February 2015 October 2014 September 2014 August 2014 April 2014 March 2014 January 2014 December 2013 November 2013 October 2013 August 2013 July 2013 June 2013 August 2012 June 2012 May 2012 February 2012 January 2012 December 2011 October 2011 August 2011 May 2011 April 2011 March 2011 February 2011 January 2011 December 2010 November 2010 October 2010 September 2010 August 2010 July 2010 June 2010 May 2010 April 2010 March 2010 February 2010 January 2010 December 2009 November 2009 October 2009 September 2009 August 2009 July 2009 June 2009 May 2009 April 2009 March 2009 February 2009 January 2009 December 2008 November 2008 October 2008 September 2008 August 2008 July 2008 June 2008 May 2008 April 2008 March 2008 February 2008 December 2007 November 2007 October 2007 September 2007 August 2007 July 2007 June 2007 May 2007 April 2007 March 2007 February 2007 January 2007 December 2006 November 2006 October 2006 September 2006 August 2006 July 2006 June 2006

• Recent comments

  • JT on How to truncate git history (sample script included)
  • Charlotte Semaan on How to fix: Xiaomi Mi Band stopped tracking steps and sleep
  • SAM on How to fix: Xiaomi Mi Band stopped tracking steps and sleep
  • SY on Convert MySQL database from one encoding/collation into another
  • Bogdan on Citation and reference managers for Linux: short overview

• Meta

  • Log in

Information criteria for choosing best predictive models

29th May 2012

Usually I’m using 10-fold (non-stratified) CV to measure the predictive power of the models: it gives consistent results, and is easy to perform (at least on smaller datasets).

Just came across the Akaike’s Infor­ma­tion Criterion (AIC) and Schwarz Bayesian Infor­ma­tion Criterion (BIC). Citing robjhyndman,

Asymp­tot­i­cally, min­i­miz­ing the AIC is equiv­a­lent to min­i­miz­ing the CV value. This is true for any model (Stone 1977), not just lin­ear mod­els. It is this prop­erty that makes the AIC so use­ful in model selec­tion when the pur­pose is prediction.
…
Because of the heav­ier penalty, the model cho­sen by BIC is either the same as that cho­sen by AIC, or one with fewer terms. Asymp­tot­i­cally, for lin­ear mod­els min­i­miz­ing BIC is equiv­a­lent to leave–v–out cross-​​validation when v = n[1-1/(log(n)-1)] (Shao 1997).

Want to try AIC and maybe BIC on my models. Conveniently, both functions exist in R.

This entry was posted on Tuesday, May 29th, 2012 at 13:44 and is filed under Bioinformatics, Machine learning. You can follow any responses to this entry through the RSS 2.0 feed. You can skip to the end and leave a response. Pinging is currently not allowed.

Leave a Reply