The use of “Big Data” to explain fluctuations in the broader economy or guide the business decisions of a firm is now so commonplace that in some instances it has even begun to rival more traditional government statistics and business analytics. Big data sources can very often provide advantages when compared to these more traditional data sources, but with these advantages also comes the potential for pitfalls. We lay out a framework called SMALL that we have developed in order to help interested parties as they navigate the big data minefield. Based on a set of five questions, the SMALL framework should help users of big data spot concerns in their own work and that of others who rely on such data to draw conclusions with actionable public policy or business implications. To demonstrate, we provide several case studies that show a healthy dose of skepticism can be warranted when working with and interpreting these new big data sources.