themodernscientist

Weekend Hacks: My Own Deep Learning Box

Michelle Gill — Sun, 11 Jun 2017 20:14:00 -0400

Deep learning is becoming my favorite topic for personal projects. Its proliferation in the last few years has produced a vast array of applications and made it much more accessible. The one aspect of it I don't love is its impact on my AWS bill. GPU instances can get costly …

Blogging O'Reilly's Artificial Intelligence Conference

Michelle Gill — Sun, 02 Oct 2016 20:14:00 -0400

Last Monday and Tuesday, I attended O'Reilly's first ever Artificial Intelligence Conference, courtesy of Women in Machine Learning & Data Science (WiMLDS). I tweeted about many of the talks I saw, but here is the recap on their blog.

Many thanks to WiMLDS for this opportunity. I will be attending O'Reilly …

Prediction of Zika Outbreaks

Michelle Gill — Wed, 28 Sep 2016 20:14:00 -0400

I did it! My final project¹ for Metis's Data Science Bootcamp was presented just over a week ago. Before the dust completely settles on this wonderful experience, I'd like to share a few of my projects with you in the coming days.

The first of these projects involved predicting …

git commit -m 'Update life'

Michelle Gill — Sun, 24 Jul 2016 20:14:00 -0400

You may have noticed some changes around here and on my Twitter account. I've relocated to my beloved New York City and am currently attending an immersive data science bootcamp at Metis. Through mid-September, I'll be honing my existing skills in Python, statistics, and analysis and developing an entirely new set in topics like machine learning, Javascript, SQL, and Hadoop.

An integral part of Metis's bootcamp is the completion of five self-designed data science projects that both emphasize and develop data science and machine learning skills. These projects use real world¹ data, tight deadlines, and simulate many of the challenges encountered by data scientists. I started my third project last Monday and look forward to sharing a few of them here and on GitHub in the coming weeks.

pdLSR: Pandas-aware least squares regression

Michelle Gill — Sun, 17 Jul 2016 20:14:00 -0400

I have a new Python project I would like to share with the community. Actually, this project isn't so new. I developed an initial version about two years before completing my postdoctoral research, and it has undergone various revisions over the past three years. Having finally made time to give it the clean-up it needed,¹ I am excited to share it on GitHub.

Overview

pdLSR is a library for performing least squares minimization. It attempts to seamlessly incorporate this task in a Pandas-focused workflow. Input data are expected in dataframes, and multiple regressions can be performed using functionality similar to Pandas groupby. Results are returned as grouped dataframes and include best-fit parameters, statistics, residuals, and more. The results can be easily visualized using seaborn.

pdLSR currently utilizes lmfit, a flexible and powerful library for least squares minimization, which in turn, makes use of scipy.optimize.leastsq. I began using lmfit because it is one of the few libraries that supports non-linear least squares regression, which is commonly used in the natural sciences. I also like the flexibility it offers for testing different modeling scenarios and the variety of assessment statistics it provides. However, I found myself writing many for loops to perform regressions on groups of data and aggregate the resulting output. Simplification of this task was my inspiration for writing pdLSR.

pdLSR is related to libraries such as statsmodels and scikit-learn that provide linear regression functions that operate on dataframes. However, these libraries don't directly support grouping operations on dataframes.

The aggregation of minimization output parameters that is performed by pdLSR has many similarities to the R library broom, which is written by David Robinson and with whom I had an excellent conversation about our two libraries. broom is more general in its ability to accept input from many minimizers, and I think expanding pdLSR in this fashion, for compatibility with statsmodels and scikit-learn for example, could be useful in the future.

Throwing the Book at Your Data

Michelle Gill — Wed, 22 Jul 2015 07:42:00 -0400

You may have noticed all has been quiet on the blog front from me lately. There are several reasons for this,¹ but I can assure you it isn't for lack of things to write about. Today, I'm happy to share with you one of the projects that has been …

Secure Password Use in Scripts

Michelle Gill — Sun, 04 Jan 2015 19:48:00 -0500

NOTE: If you're using passwords in shell scripts, see Daniel Jalkut's reply below about using OS X's built-in security command instead of his AppleScript library.

One of my goals for the new year¹ is to be more security minded in my use of technology. This is a broad statement …

Encryption of BitTorrent Sync Peers

Michelle Gill — Fri, 21 Feb 2014 19:48:00 -0500

I'd like to follow up on a previous post covering BitTorrent Sync. In this post, I mentioned privacy is one of the advantages afforded by BitTorrent Sync over traditional cloud storage options, such as Dropbox. Like most cloud storage options, data are encrypted during transmission with BitTorrent Sync.¹ Once …

Torrential File Synchronization

Michelle Gill — Thu, 06 Feb 2014 19:48:00 -0500

My research requires that I work in multiple locations: in the lab, at a multi-institution shared instrument facility, and sometimes in my fuzzy slippers at home. Having my data and processing scripts updated on whatever computer I happen to be using is critical to my ability to get work done …

Easy Syntax Highlighting With Pygments

Michelle Gill — Thu, 24 Oct 2013 15:19:00 -0400

As a scientist who splits her time between both wet lab and computational work, I have learned the importance of saving code snippets for future reference. These snippets are handy when I later can't recall a clever Pandas indexing trick or when I want to share code with a colleague …

Automated Notifications of Experiment Progress: Prowl Extra Credit

Michelle Gill — Sat, 19 Oct 2013 15:19:00 -0400

Previously, I posted about using Growl in combination with Prowl to get remote notifications of experiment progress on both a Mac and iPhone. Later that day, I started thinking about some improvements to the script after a brief Twitter conversation with Seth Brown.¹

The script depends on the remote …

Automated Notifications of Experiment Progress: Combining Shell, SSH, Growl, and Prowl

Michelle Gill — Sat, 12 Oct 2013 15:19:00 -0400

In addition to covering my use of python in research, one of my goals for this blog is to share ways I use various other computational tools to automate basic research tasks. Such an opportunity arose this week with the onset of hardware issues causing one of our laboratory's NMR …

Visualization of NMR Shaped Pulses: Fun with Javascript Animation

Michelle Gill — Fri, 14 Jun 2013 15:35:00 -0400

This IPython notebook builds on the previous blog post which described how to simulate and plot the result of a shaped pulse on magnetization in an NMR experiment. For the purpose of teaching¹, it can also be useful to visualize the effect of this pulse on the magnetization at …

Simulation of NMR Shaped Pulses: NumPy vs Fortran

Michelle Gill — Sun, 09 Jun 2013 15:35:00 -0400

Bandwidth-selective excitation in NMR is commonly accomplished through the use of shaped pulses. These pulses require careful calibration to ensure power levels and pulse lengths are correctly determined for optimal excitation of only the desired frequency range. Simulating the shaped pulse over a range of frequencies is one way of …

My Octopelican Python Blog

Michelle Gill — Sun, 02 Jun 2013 15:48:00 -0400

Many have asked about my blog design and how I incorporate IPython notebooks, such as the one used in the previous post. I merely glued all the pieces together for this setup, so to give credit where it is due and to provide a documented how-to for my future self …

Binary Integer Programming With Python

Michelle Gill — Wed, 29 May 2013 20:14:00 -0400

My research is focused on biomolecular NMR and, as such, often involves transferring resonance assignments from one multidimensional NMR spectrum to another. In theory this should be a simple task, but it can be extremely time consuming due to small variations in the sample used to acquire the data or …

Please Update Your RSS Subscription

Michelle Gill — Sun, 26 May 2013 13:25:00 -0400

You are receiving this post because I’ve migrated my RSS feeds from FeedBurner to URI.LV. I attempted to automatically migrate all subscribers to the new feed but, due to consequences outside my control, that wasn’t possible. So, I’d greatly appreciate it if you could resubscribe using …

For Some, This is a Busy Week

Michelle Gill — Sun, 07 Apr 2013 23:14:00 -0400

The coming week is going to be an incredibly busy, but exciting, one for me. Besides having updated my blog so that I am now one of the cool kids with a Pelican-generated blog hosted at GitHub,¹ the following things are also taking place:

To My Friend

Michelle Gill — Sat, 16 Mar 2013 19:19:00 -0400

This is, in so many ways, not the topic I envisioned for my second post. Sometimes life has other plans.

Hello, Part Deux

Michelle Gill — Sun, 16 Dec 2012 15:19:00 -0500

After much consternation and hand wringing, I am attempting to revive my blog. I've learned from previous deterrents and now have a clearer vision of the topics I'd like to cover. So, for the time being, I am simply saying "hello" and giving those interested the opportunity to subscribe.

In …