Year: Winter Session II 2019
Time: Wed & Fri: 5:00pm-6:30pm
Location: iSchool Greg Lab. (Irving K. Barber, 4th floor)
Instructor: Dr. Muhammad Abdul-Mageed
Office location: School of Information (iSchool) 494
Office phone: 6048-274-530
Office hours: Wed. 11:00am - 1:00pm or by appointment. (I can also handle inquiries via email or Skype.)
E-mail address: [email protected]
Student Portal: http://canvas.ubc.ca
Rationale/Background: Deep learning, a class of machine learning methods inspired by information processing in the human brain, has revolutionized the way we build machines, automate processes, analyze data, and just problem-solve in a fast-increasing host of domains. These transformational changes have disrupted whole industries, and are expected to impact wide sectors of society. Scholarship in the field has been growing very rapidly, with significant funding provided for both basic research and applications.
Goal: This is a graduate-level, directed resarch course aimed at bridging the gap between basic deep learning knowledge and novel and bleeding-edge deep learning research. The course will emphasize deep learning methods developed over the past 1-2 years, with more attention allocated to the last 8-12 months of scholarship in the field. Areas will include (some of) the below:
-
Generative Deep Learning
-
Attention
-
Representation Learning
-
Bayesian Deep Learning
-
Deep Learning over Graphs
Students can choose any area of application, including:
-
Unsupervised Machine Translation
-
Deep Learning for Brain Signal
-
(Cross-Lingual) Natural Language Understanding
-
Grounding; Language and Visual Intelligence
-
Style Transfer
-
Automatic Summarization
-
Misinformation Detection over Social Networks
Potential audiences for this course are:
- Students with a deep learning background from any field. This includes people who have taken LING530F (Deep Learning for Natural Language Processing) and those who have taken a graduate-level ML course and are already familiar with deep learning methods. (This course will not cover deep learning basics. See also pre-requisites below).
Upon completion of this course students will be able to:
-
identify a sub-area of deep learning research, engineer a solution, and write a high-quality research paper for a top venue.
-
prepare and deliver a presentation based on a novel deep learning research area or paper.
-
understand a range of deep learning topics at a research level theoritically, and develop related engineering solutions
TBA (But see above). There will be some space to customize some select sessions based on student interests which might not be covered in areas above.
- Knwoledge of basic deep learning methods (as demonstrated by course work or at least one publication). Students who have taken [Deep Learning for Natural Language Processing] satisfy pre-requisites. Students vetted by a) a Professor or b) another student who took [DLNLP] will be allowed to register.
• This is not a regular course. Rather, this is a directed research/reading course that will operate in a seminar fashion where students and the instructor meet twice a week to read and discuss novel research papers. Some sessions will be focused on engineering. Presentations will rotate among students, with enough prior time for preparation. This course has no homework. Output will be a research paper by the end of the semester.
Required books:
- Goodfellow, I., Bengio, Y., Courville, A., & Bengio, Y. (2016). Deep learning (Vol. 1). Cambridge: MIT press. Available at: [link].
Note: Will be arranged further upon organizational meeting/discussion in class.
- [Neural Sentiment Classification with User and Product Attention]
- [A Lexicon-Based Supervised Attention Model for Neural Sentiment Analysis]
- [Weakly Supervised One-Shot Detection with Attention Siamese Networks]
- [Graph-Attention Networks]
- [Unsupervised machine translation using monolingual corpora only]
- [Phrase-Based & Neural Unsupervised Machine Translation]
- [Towards Unsupervised Speech-To-Text Translation]
- [Rapid adaptation of neural machine translation to new languages]
- [Bidirectional Generative Adversarial Networks for Neural Machine Translation]
- [Style transfer as unsupervised machine translation]
- [Understanding back-translation at scale]
- [Back-translation-style data augmentation for end-to-end ASR]
- [Denoising Neural Machine Translation Training with Trusted Data and Online Data Selection]
- [Unsupervised Text Style Transfer using Language Models as Discriminators]
- [Multiple-Attribute Text Style Transfer]
- [Learning Criteria and Evaluation Metrics for Textual Transfer between Non-Parallel Corpora]
- [Unsupervised Controllable Text Formalization]
- [Improving language understanding by generative pre-training]
- [Content preserving text generation with attribute controls]
- [Adversarial Text Generation via Feature-Mover's Distance]
- [Generating sentences by editing prototypes]
- [Texygen: A Benchmarking Platform for Text Generation Models]
- [Eval all, trust a few, do wrong to none: Comparing sentence generation models]
- [Maskgan: Better text generation via filling in the _]
- [On accurate evaluation of gans for language generation]
- [Neural text generation: Past, present and beyond]
- [Recipe1M: A Dataset for Learning Cross-Modal Embeddings for Cooking Recipes and Food Images]
- [Unsupervised Image Captioning]
- [Learning to Read by Spelling: Towards Unsupervised Text Recognition]
- [Extractive Summary as Discrete Latent Variables]
- [Middle-Out Decoding]
- [Language GANs Falling Short]
- [A deep learning method for classification of EEG data based on motor imagery]
- [Converting your thoughts to texts: Enabling brain typing via deep feature learning of EEG signals]
- [Brain2Object: Printing Your Mind from Brain Signals with Spatial Correlation Embedding]
- [Learning Temporal Information for Brain-Computer Interface Using Convolutional Neural Networks]
- [Validating deep neural networks for online decoding of motor imagery movements from EEG signals]
- [Understanding disentangling in β-VAE]
- [Isolating Sources of Disentanglement in VAEs]
- [A Tutorial on Deep Latent Variable Models of Natural Language]
- []
- []
| Assignment | Due date | Weight |
|---|---|---|
| Professionalization & Class Participation | Throughout | 10% |
| First presentation | 10% | |
| Second presentation | 10% | |
| Third presentation (optional; grade can be allocated to final project) | 10% | |
| Individual or Group assignment: Project Proposal | Ungraded | |
| Individual or Group assignment: Term Project | 60% |
Presentations will be graded with a check mark upon completion, meaning students will be assigned full presentation grade upon delivery. Since this is a directed reading/research course, students are expected to do their best to deliver high quality presentations. This includes making sure a student starts preparing early, does her/his best to digest and explain the matrial, and include enough examples to illustrate a concept. Concievably, some material will be techincal and/or will need enough attention. Discussions in class will be complementary to the student presentations and all students will be expected to participate by asking questions and possibly answering others' questions..
**Deliverables:
A presentation (with slides and possibly code)
Students will prepare a 2-page proposal of a research project. The proposal should have enough details about the method, data, brief lietrature review, expected outcome, and (in group projects) a breakdown of work over time (and among members).Students will have the opportunity to discuss their ideas closely with the instructor and their peers.
**Deliverables:
A 2-page project proposal.
The major assignment in this course is a research paper that presents a novel deep learning method to solve a problem, or a novel application of one of the deep learning methods covered in class. The paper should be of high-quality and follow the norms of academic publishing within the DL and/or NLP community. Students will have the opportunity to work closely with the isntructors and peers on their projects.
**Deliverables:
An 8-page, publishable, research paper + all relevant code.
Attendance: Since this is a directed research/reading course, some Friday sessions may be canceled upon agreement in class, with prior notice. Otherwise, students are expected to attend regularly and come prepared. The UBC calendar states: “Regular attendance is expected of students in all their classes (including lectures, laboratories, tutorials, seminars, etc.). Students who neglect their academic work and assignments may be excluded from the final examinations. Students who are unavoidably absent because of illness or disability should report to their instructors on return to classes.”
Evaluation: All assignments will be marked using the evaluative criteria given in this syllabus.
Access & Diversity: Access & Diversity works with the University to create an inclusive living and learning environment in which all students can thrive. The University accommodates students with disabilities who have registered with the Access and Diversity unit. You must register with the Disability Resource Centre to be granted special accommodations for any on-going conditions. Religious Accommodation: The University accommodates students whose religious obligations conflict with attendance, submitting assignments, or completing scheduled tests and examinations. Please let your instructor know in advance, preferably in the first week of class, if you will require any accommodation on these grounds. Students who plan to be absent for varsity athletics, family obligations, or other similar commitments, cannot assume they will be accommodated, and should discuss their commitments with the instructor before the course drop date. UBC policy on Religious Holidays.
Plagiarism Plagiarism is the most serious academic offence that a student can commit. Regardless of whether or not it was committed intentionally, plagiarism has serious academic consequences and can result in expulsion from the university. Plagiarism involves the improper use of somebody else's words or ideas in one's work.
It is your responsibility to make sure you fully understand what plagiarism is. Many students who think they understand plagiarism do in fact commit what UBC calls "reckless plagiarism." Below is an excerpt on reckless plagiarism from UBC Faculty of Arts' leaflet, Plagiarism Avoided: Taking Responsibility for Your Work.
"The bulk of plagiarism falls into this category. Reckless plagiarism is often the result of careless research, poor time management, and a lack of confidence in your own ability to think critically. Examples of reckless plagiarism include:
- Taking phrases, sentences, paragraphs, or statistical findings from a variety of sources and piecing them together into an essay (piecemeal plagiarism);
- Taking the words of another author and failing to note clearly that they are not your own. In other words, you have not put a direct quotation within quotation marks;
- Using statistical findings without acknowledging your source;
- Taking another author's idea, without your own critical analysis, and failing to acknowledge that this idea is not yours;
- Paraphrasing (i.e. rewording or rearranging words so that your work resembles, but does not copy, the original) without acknowledging your source;
- Using footnotes or material quoted in other sources as if they were the results of your own research; and
- Submitting a piece of work with inaccurate text references, sloppy footnotes, or incomplete source (bibliographic) information."
Bear in mind that this is only one example of the different forms of plagiarism. Before preparing for their written assignments, students are strongly encouraged to familiarize themselves with the following source on plagiarism: the Academic Integrity Resource Centre.
If after reading these materials you still are unsure about how to properly use sources in your work, please ask me for clarification. Students are held responsible for knowing and following all University regulations regarding academic dishonesty. If a student does not know how to properly cite a source or what constitutes proper use of a source it is the student's personal responsibility to obtain the needed information and to apply it within University guidelines and policies. If evidence of academic dishonesty is found in a course assignment, previously submitted work in this course may be reviewed for possible academic dishonesty and grades modified as appropriate. UBC policy requires that all suspected cases of academic dishonesty must be forwarded to the Dean for possible action.