CS 529A: Data Mining (and Applications to Computational Biology) - Spring 2004

Instructor:  Weixiong Zhang
Prereq:  CS241 and SSM 326A (or Math 320), or their equivalent, or permission of the instructor
Text book:

• D. J. Hand, H. Mannila and P. Smyth, Principles of Data Mining, MIT Press, 2001

Reference books:

• J. Han and M. Kamber, Data Mining: Concepts and Techniques, Morgan Kaufmann, 2001
• T. Hastie, R. Tibshirani and J. Friedman, The Elements of Statistical Learning, Springer, 2001.

Location: Whitaker Hall 216
Time: Monday and Wednesday, 1:00 pm - 2:30 pm
Office hours: Monday and Wednesday, 2:30 - 3:30 pm, Jolley Hall 506, or by appointment.

Brief description:  Many scientific computing problems are, by nature, statistical.  Such problems appear in many domains, such as text analysis, data mining on the web, computational biology and various medical applications.  Another source of the statistical nature of such problems is the lack of sufficient information of the problem domains as well as the specific problems at hand.  What is available for a typical application is usually a set of data from observation or experiments.  The main objective of this course is to gain experience of dealing with statistical data analysis problems by studying various statistical methods that can be used to make sense out of data, by reading and reviewing literature as well as by working on a specific statistical problem in a selected application domain.

Syllabus
Reading materials on biology
Policies on homework, project, and grading
Collaboration Policy
Homework assignments

Syllabus  (Note: This syllabus will be adjusted as needed to meet the needs of the course.)

******** Introduction and background ********

• Topics: What data mining is about; Characteristics and relationship with statistics and machine learning; Tasks, objectives, components (Models, patterns, and evaluation, and algorithms) and overall procedures; The curse of dimensionality;  Uncertainty in data.  Basic background of biology.

• Dates:  1/21, 1/26, 1/28

• Reading: chp 1.4, 1.5, 1.6, 2.6, 2.7, 4.2, 4.7.

******** Main topics: Probability/statistical background, statistical and Bayesian Learning ********

• Topics: Basics on subjective probabilities, statistics, maximum likelihood, Bayesian estimator, independence, Bayesian networks, simulation method and Monto Carlo Markov chains, statistical modeling; Bayesian classifier; Learning Bayesian networks.

• Applications: Bayesian networks for gene expression data analysis
  

• Dates:  2/2, 2/4, 2/9, 2/11, 2/16

• Reading: N. Friedman, et al., Using Bayesian networks to analyze expression data, J. Computational Biology, 7(3/4), 2000, pp. 601-620, which can be downloaded from a machine on campus.
  

******** Main topics: Optimization and combinatorial search ******** 

• Topics: Systematic search: Dynamic programming, state-space search (best-first search and depth-first branch-and-bound); local search strategies; parameter optimization methods; optimization with missing data (EM) and Monte Carlo Markov Chain sampling.

• Applications: Global and local alignment, motif finding (Gibbs sampler)
  

• Dates:  2/18, 2/23, 2/25, 3/1, 3/3

• Reading:

******** Main topic: Classification ********

• Topics: Classification modeling; Decision-tree classification and entropy minimization; Cross validation;  Kernel methods and support vector machines
  

• Applications: Regulatory module identification; Identification of reguatory networks; Gene splicing prediction

• Dates:  3/15, 3/17, 3/22, 3/24, 3/29

• Reading:

******** Main topic: Regression ******** 

• Topics: Linear regression; Artificial neural networks.

• Applications: Identification of reguatory networks

• Dates:  3/31, 4/5, 4/7

• Reading:

******** Main topic: Clustering ******** 

• Topics: Cluster analysis; Hierarchical clustering; K-means and nearest neighbor methods; Biclustering; Probabilistic clustering

• Applications: Gene expression analysis

• Dates:  4/14, 4/19, 4/21, 4/12

• Reading:

******** Main topics:  Structure-based methods ********

• Topics: Regular and context free grammar and hidden Markov models (HMMs)

• Applications: HMM models and gene prediction; Learning a grammar from sequences
  

• Dates:  if we have time
  

• Reading:

******** Course projects ********

• Topics: Student presentations

• Dates:  4/26, 4/28, 5/3
  

Supplemental reading materials on biology

If you need basic knowledge on cell biology, microarray technology and microarray data analysis, the following links will be useful.

• H. Lodish, et al., Molecular Cell Biology (online version)

• DNA Microarray (Genome Chip)

• Bibliography on microarray data analysis
  

Specific reading materials can be found at the reading lists in the Syllabus.

Policies on homework, project and grading

• Homework Assignments: There will be 4 sets of homework assignments, one each on probability/statistics/Bayesian networks, optimization, classification/regression and clustering. The assignments are due either in the instructor's mailbox in Bryan 509 by 12:50 pm or in class at the beginning of the class on the given due date. Any homework submitted in class after 1:30pm will be given a 15 point late penalty. If you arrive to class after 1:30, wait until the end of class to bring up your homework. It is VERY disruptive to have students walking in late and coming to the front of the class to submit their homeworks. No assignments will be accepted after the instructor leaves the classroom. You are strongly advised to get started early so that you can get help if needed! These homeworks are not designed to be done in just a few days. There may not be a single way for a given problem. You are expected to spend ~10 hours for a two week homework assignment --- It will be very frustrating if you try to do this all in the last few days. For some problems you may need to think about them a while and then set them aside for a little.
• Project: There will be no exam for the course. Instead, we will have a course project for everyone. There will be two types of projects:

• Reading some papers on a particular datamining topic that we do not have time to cover or we do not go into detail in the class, and then giving an in-depth presentation to describe and discuss the problem, dataming objectives, data used, datamining procedure and techniques adopted, your criticism on the existing work, and your thoughts and suggestions on possible future research.

• Designing an algorithm/method for a particular datamining problem or compare some existing methods for the problem. 

• Computation of the Final Grade: The following elements and scores will go into the final grade:
• Course participation: 1/2 point for each class attend.  (Because the instructor will cover materials not in the textbook, it is critical to attend every lecture.)  There are a total of 28 lectures (including student presentations), you get 14 points by simply attending all the classes.
  
• Homework: 40 points total, 10 for each set of homework assignments.
  
• Project:  30 points total.  10 for presentation and 20 for report.
• Note scribing:  For each lecture, two or three students will be assigned to take detailed notes and combine them into a complete one.  The purpose is to produce a set of lecture notes for everyone at the end of the course (or each lecture) and the students who will take this course next year.  You get 5 points for each participation.
• Homework grading:  We do not have a TA for this course this semester, as it is the first time it is offered.  You are asked to help grading one or two homework assignments.  The instructor will provide you one set of possible solutions and go over with you the principle you should apply in grading.  You get 10 points for each participation.
  

Policy on collaboration

Please keep any discussions you have with other students to a small group of no more than 3 students and be sure that each of you are equally involved. If you just listen in and are then able to understand and write up the solution you have missed at least half of the benefit of the homework. It is really important to work through the process of recognizing when you are heading the wrong way and learning how to work through the problem solving process.

Violations of any of the above rules will be dealt with harshly! The homework problems and projects are designed to help you learn the material being taught. Being told the solution and understanding it is VERY different from working through the process of actually finding a solution. If you do not take an active role in the process of solving the homework problems and project, then you won't get much out of it, hence you won't learn the material.

Homework assignments

• Homework #1: Due March 1 in class or before class.

Created by Weixiong Zhang, Jan. 2004.  Last modified by Weixiong Zhang, Feb. 2004.