# CS671: Randomized Algorithms

Most recent message posted: 12/02/2015.

• Time and Location: Tuesday and Thursday, from 2:00-3:50, in room KAP 148.
• Instructor: David Kempe (Office hours: by appointment)
• TA: Yu Cheng (Office hours: Monday, 2:00-3:00 in SAL 246, or by appointment)

### Scope of Course

For many important problems, randomized algorithms are the fastest or simplest known algorithms, often both. Hence, randomization has become one of the central paradigms of modern algorithm design. At the same time, randomization is commonly used in modeling the real world, leading to a detailed study of random structures. Not only are models including randomness important in their own right, but they also often serve to prove the existence of structures previously not known to exist (Probabilistic Method). This class will cover a variety of the techniques for analyzing randomized algorithms and models of random structures. Particular emphasis will be placed on efficient or novel algorithms.

### Topics

The following is a preliminary list of topics that will be covered in terms of techniques. It may be expanded or shortened based on time availability.

• Expectation, Variance, simple bounds, and applications
• Tail Bounds (Chernoff/Hoeffding, Martingale) and applications
• MiniMax Principle and applications
• Probabilistic Method and Random Graphs
• Markov Chains, Rapid Mixing, Sampling and Counting
• Sampling and VC dimenstion

### Detailed Topic Order

Here is a more detailed list of the topics and book chapters as we go through the semester. In particular, this may help students read ahead.

1. Overview and basics of probability theory
2. Intersection of events and the Randomized Min-Cut Algorithm (Sections 1.1, 10.2)
3. Unions of Events and random partitions
4. Linearity of Expectations and Max-SAT (Section 5.2)
5. Randomized Quicksort and Median Finding (Sections 1.1, 1.4)
6. Lower Bounds and Game Tree Evaluation (Sections 2.1, 2.2)
7. Derandomization (Sections 2.3, 5.6)
8. Variance and its uses (Sections 3.1, 3.2, 3.3)
9. Chernoff Bounds and Applications (Sections 4.1, 4.2, 4.3)
10. Martingale Sequences and Applications (Section 4.4 and additional material)
11. VC dimension and applications (additional material)
12. Fingerprinting techniques (Sections 7.1-7.6, 12.4)
13. Probabilistic embedding of metric spaces (additional material)
14. Markov Chains and Random Walks (Chapters 6, 11, additional material)
15. Lovasz Local Lemma and a constructive version (Section 5.5 and paper by Moser/Tardos).
16. Randomness Amplification (Sections 3.4, 5.3, 6.8)

### Prerequisites

• CSCI 570 (grade at least A-), or CSCI 670 (grade at least B) or explicit permission of instructor.
• familiarity with mathematical reasoning, in particular probability theory (sample spaces, random variables, expectation, higher moments).

The following textbook is required:
• R. Motwani and P. Raghavan: Randomized Algorithms (Cambridge University Press)

In addition, we will occasionally reference recent (and not-so-recent) research or survey papers. Other useful sources of information (in some cases covering material beyond the scope of this course) are the following:

• M. Mitzenmacher and E. Upfal: Probability and Computing (Cambridge University Press).
• N. Alon and J. Spencer: The Probabilistic Method (Wiley).
• M. Habib, C. McDiarmid, J. Ramirez-Alfonsin and B. Reed (Editors): Probabilistic Methods for Algorithmic Discrete Mathematics (Springer).
For reviewing prerequiste material, see:
• J. Kleinberg and E. Tardos: Algorithm Design (Addison-Wesley).
• S. Dasgupta, C. Papadimitriou, U. Vazirani: Algorithms (McGraw Hill).
• T. Cormen, C. Leiserson, R. Rivest and C. Stein: Introduction to Algorithms (MIT Press).
The following papers (and possibly more) will be covered in depth in class:

There will be about 4-5 homework assignments, one takehome midterm, and one takehome final. The homeworks will cumulatively count for 40% of the grade, the midterm for 25%, and the final for 35%. All assignments will be posted here once issued.

Homeworks should be predominantly done individually. However, limited interaction is appropriate, so long as it is clearly marked on the homework who you discussed ideas with. See more detail below.

The midterm and final must be solved individually. You cannot even discuss basic ideas with classmates or anyone else. Not even "Oh, you just use Chernoff Bounds", or "Question 3 seems pretty difficult." If you have any questions, ask the instructor only.

### Statement on Academic Conduct and Support Systems

Plagiarism - passing someone else's ideas as your own, either verbatim or recast in your own words - is a serious academic offense with serious consequences. Please familiarize yourself with the discussion of plagiarism in SCampus in Section 11, Behavior Violating University Standards https://scampus.usc.edu/1100-behavior-violating-university-standards-and-appropriate-sanctions/. Other forms of academic dishonesty are equally unacceptable. See additional information in SCampus and university policies on scientific misconduct, http://policy.usc.edu/scientific-misconduct/.

Here's more concretely what is and isn't allowed in CSCI 671. During the exams, there must be no collaboration. While you have more time and get to do the exam in the comfort of your own home, for the purpose of interaction, pretend that you are taking the exam in a standard exam setting. So absolutely no communication about anything related to the class with anyone.

For homeworks, limited collaboration is allowed - after all, research itself is a collaborative process, and many of the questions you will be working on are recent research results. In assessing whether your collaboration is going too far, the rule you should follow (literally) is the following: after you had a discussion with someone, you destroy all notes/photos/whiteboard markings from your discussion, and spend at least one hour doing something completely different. Then, you write up your own solution, based on your memory and your ability to rederive it. You should never look at a classmate's solution, or show your (finished or nearly finished) solution to a classmate. Yes, even if you show your solution to someone else, you are cheating. And of course, whenever you collaborate on ideas, you should give credit and mark prominently on your submission who you collaborated with for which problems. That way, I'll know that you are not trying to hide something.

My policy is that for any exam/assignment on which you cheated, your grade will be 0, and the case reported to the university. Perhaps more importantly, if you cheat, I will tell your advisor, and he/she will likely kick you out of your research group, since integrity is extremely important for a budding researcher.

Discrimination, sexual assault, and harassment are not tolerated by the university. You are encouraged to report any incidents to the Office of Equity and Diversity http://equity.usc.edu/ or to the Department of Public Safety http://capsnet.usc.edu/department/department-public-safety/online-forms/contact-us. This is important for the safety whole USC community. Another member of the university community - such as a friend, classmate, advisor, or faculty member - can help initiate the report, or can initiate the report on behalf of another person. The Center for Women and Men http://www.usc.edu/student-affairs/cwm/ provides 24/7 confidential support, and the sexual assault resource center webpage sarc@usc.edu describes reporting options and other resources.

#### Support Systems

A number of USC's schools provide support for students who need help with scholarly writing. Check with your advisor or program staff to find out more. Students whose primary language is not English should check with the American Language Institute http://dornsife.usc.edu/ali, which sponsors courses and workshops specifically for international graduate students. The Office of Disability Services and Programs http://sait.usc.edu/academicsupport/centerprograms/dsp/home_index.html provides certification for students with disabilities and helps arrange the relevant accommodations. If an officially declared emergency makes travel to campus infeasible, USC Emergency Information http://emergency.usc.edu/ will provide safety and other updates, including ways in which instruction will be continued by means of blackboard, teleconferencing, and other technology.

### Messages

• 12/02/2015: The takehome final is posted. It is due in David's office by 3:00pm on Thursday, 12/10. As with the midterm, no collaboration of any kind is allowed, and the final cannot be submitted late.
• 11/23/2015: The fifth homework assignment is posted. It is due in class on Thursday, December 03. This assignment also contains a problem from the textbook. Again, you must not seek out solutions to this problem from online or other sources.
• 11/16/2015: Question 4 of HW4 as stated (the lower bound part) was wrong. The assignment has been updated with a revised version, asking you to prove the bound in the L1 norm rather than L2.
• 11/08/2015: The fourth homework assignment is posted. It is due in class on Thursday, November 19. Notice that this assignment contains some problems from the textbook which give pointers to papers that solve these problems. It is of course not appropriate to seek out those solutions.
• 10/25/2015: The midterm has been updated to clarify and fix Question 2.
• 10/22/2015: The takehome midterm has been posted. It is due in class on Thursday, 10/29. You cannot submit it late.
• 10/19/2015: HW3, linked below, and its due date, have been updated to account for the mistake in Question 4.
• 10/07/2015: The third homework assignment is posted. It is due in class on Tuesday, October 22.
• 10/05/2015: Yu Cheng will not hold office hours on 10/19.
• 10/01/2015: There will be no class on 10/13 or 10/15, as I will be at a workshop in Berkeley.
• 09/17/2015: The second homework assignment is posted. It is due in class on Tuesday, September 29.
• 08/30/2015: The first homework assignment is posted. It is due in class on Tuesday, September 8.
• 08/19/2015: This is where announcements about homeworks and other class-related issues will be posted. Check back frequently. (Note that the Blackboard Page will not be used.)