''Arranged around the theme of Economics in the Age of Algorithms, Experiments and A.I.''
Share this page:
Anyone who knows me would be taking a safe bet that I wouldn’t miss this. While I was only recently introduced to NABE and more generally the world of Tech Economics, I could not be more excited for this conference. I had very high expectations and I must admit that TEC2019 definitely surpassed them. I got to see up close some of the biggest names in various of my fields of interest like the design of reputation systems with Michael Luca, Dina Mayzlin and Steven Tadelis discussing their work and recent advances in the area. I met and interacted with tech industry experts from Amazon, Zillow, Facebook, Google and various other tech companies in pre-organized small-group dinners and cocktail receptions and met other PhD students interested in similar topics. The only downside? Concurrent sessions meant that I had to forgo a lot of potentially interesting talks! It was 2.5 days filled with information, positivity and excitement. Why does tech want economists and why do economists want to be in tech? How does the economist’s skillset and thinking blend in with other tech disciplines like data science and engineering? How does behavioral economics blend in tech? What have economists done so far in tech and what’s their future in tech? These and many other similar questions were discussed in various sessions, luncheons and cocktail receptions.
In what follows, I will try to provide brief summaries of most of the sessions that I attended as well as some pictures from the conference. My summaries are definitely incomplete and provide only a small glimpse of the talks. Find the slides from most presentations at TEC2019 here.
Session: Outlook for Economist Careers in Tech
Speakers: Peter Gross, Senior Director, Economic Data and Product, Zillow Group; Steve McBride, Director of Data Science and Engineering, Content Strategy and Economics, Netflix; Ana-Maria Mocanu, Data Science Manager, Stripe.
Moderator: Carolyn Evans, Chief economist at Intel.
Talking points and key takeaways:
- Cooperation across multiple disciplines.
- Constructive feedback.
- Structured thinking is a critical element that economists bring to the table even though structural estimation is not common.
- Economists generally lack technology skills that data scientists posses and perhaps PhD Economics programs should address that.
Session: Technology and Cities
Speakers: Ed Glaeser, Fred and Eleanor Glimp Professor of Economics, Harvard University.
Introduction: Jim Diffley, CBE, Senior Director – Economics, HIS Markit.
Talking points and key takeaways:
- The impact of Uber, Lyft, Zipcar and Yelp.
- The problem with affordable housing.
- The importance of having “nice neighbors”
- How can technology help?
- Measure of road quality through phone bumpiness.
- Analysis of the returns in improving road quality.
- The promises of working remotely.
Session: Diversity in Tech and Economics
Speakers:Susan Athey, The Economics of Technology Professor, Stanford Graduate School of Business; Tracy Chou, Co-Founder, Project Include; Brent Turner, Chief Operating Officer, Rover.
Moderator: Clementine Van Effenterre, Assistant Professor, Department of Economics, University of Toronto.
Talking points and key takeaways:
- A discussion on the improvements and the challenges that lie ahead in improving diversity through inclusion in our community.
- Brent Turner talked about how he built Rover with the intent of making it a company where women would love to work at. He mentioned that he believe that a consequence of that would be that everyone would love to work there.
- Susan Athey shared a story about interviews at AEA meetings being held in hotel rooms
- Tracy Chou talked about literally coding a block button to block an online bully and mentioned a funny story about censoring an article about penis length versus girth.
Session: Experiments and Machine Learning
Speakers: Susan Athey, The Economics of Technology Professor, Stanford Graduate School of Business; Dominic Coey, Economic Research Scientist, Facebook Research.
Introduction: Moshe Katzwer, Data Scientist, Uber.
Talking points and key takeaways:
- Susan Athey discussed the AB testing paradigm and how to model the performance of control units in two papers “Designing experiments with staggered rollouts” and “Confidence intervals for policy evaluation in adaptive experiments”
- Dominic Coey discussed the three areas of machine learning with examples:
- Supervised learning – Learning relationships between inputs and outputs from past example data. Application: “Compliance-weighting” for variance reduction.
- Unsupervised learning – Finding latent structure in the data. Application: Graph partitioning for cluster randomization in experiments with spillovers.
- Reinforcement learning – Generating good behavioral policies in stochastic, dynamic environments. Application: Importance sampling for counterfactual evaluation of ML models.
- A key takeaway is to stop asking people questions that the platform already has a pretty confident answer to. Instead use people’s limited time and attention to ask questions that are of value but have less data on thus maximizing the usefulness of feedback.
Session: Tools from Cloud Computing Every Economist Should Know
Speakers: Ashish Mishra, Principal Engineer, CoreAI team Amazon.
Introduction: Duncan Gilchrist, Pricing and Incentives, Uber.
Talking points and key takeaways:
- AWS, Hadoop and Could data catalog.
- MapReduce, Apache Flink, Beam, Apache Spark.
- Cloud computing improves scalability by parallelizing over many machines however there are drawbacks: Network, Disk, Memory and Threading.
- Ended by discussing the benefits of version control systems.
Session: All Companies Are Tech Companies Now
Speakers: Alexei Alexandrov, Director, Wayfair; Jon Gurney Bell, Advanced Technology Group, UPS; Lester Jones, CBE, Chief Economist, National Beer Wholesalers Association; Chris Norby, Deputy Director of Analytics, Boeing Global Services.
Moderator: Carolyn Evans, Chief Economist, Intel.
Talking points and key takeaways:
- This was a relaxed, fun and eye-opening session where industry experts from companies and industries one would not expect to see in a tech-economics conference, took turns to explain how tech economics is revolutionizing their field and how their respective companies are adjusting and benefiting.
- It was interesting to find out that Boeing aircrafts have more lines of code than Microsoft’s API. Chrys Norby talked about automation and smart systems in manufacturing, parallel computing, Boeing analytics, model-based simulations, VR and AR.
- Lester Jones discussed how technology and the highly regulated market has leveled the playing field for large and smaller firms in the beer industry of declining consumption but increasing value and competition.
- Finally, Jon Gurney Bell discussed the use of drones, how more efficient routing algorithms can save $50M if they can reduce 1 mile per driver per day. He finished his talk right on time.. just like a UPS delivery according to him!
Luncheon: What Can Machine Learning Do and What Does it Mean for the Economy?
Speakers: Erik Brynjolfsson, Professor, MIT Sloan School of Management; Director, MIT Initiative on the Digital Economy.
Introduction: Sara Rutledge, Managing Director of Real Estate Products, StratoDem.
Talking points and key takeaways:
- Improvements in accuracy are leading to a switch between human experts to machine predictors for example in medical image recognition.
- AI is bringing about a big job transition rather than job loss, even though in the short term it might seem like the former.
- Humans are uniquely equipped to do some types of tasks and jobs that AI won’t take over for decades or centuries.
- New tasks in the Machine Learning world are the redesign of tasks and how to ask better questions with this powerful tool.
- Besides all the advances in ML and AI, there has not been much productivity boom. This is similar with many new technologies that takes decades for industries to redesign their production lines and factories to take full advantage.
- However there is a projected productivity boom over the coming decade.
- The biggest challenge is that there exists no economic law saying that everyone will benefit from the increase in productivity. It is very possible that only a few will benefits and others may lose out.
Session: Designing Reputation Systems
Speakers: Dina Mayzlin, Professor of Marketing, USC Marshall School of Business; Steve Tadelis, Amazon Economist Fellow, AWS Business Systems / Professor of Economic, Business and Public Policy, Haas School of Business, UC Berkeley; Michael Luca, Lee J. Styslinger III Associate Professor of Business Administration, Harvard University.
Moderator: Alexei Alexandrov, Director, Wayfair.
Talking points and key takeaways:
- This was my most anticipated session since online reputation systems are at the core of my job market paper.
- What are the social preferences that cause people to review and consequently what are the systematic biases in online reviews. As such how can we design solutions around these biases.
- The impact of the Yelp reputation system and that showing just the average rating is not good enough. How can we use other metrics beyond reviews to improve upon the review data.
- The impact of fake reviews and how the better and more powerful become the reputation systems, the higher are the incentives of sellers to try and game the system.
- The cold-start problem (how to get people to buy your product when you have no or little reviews) and how reputation can act as a barrier to entry.
- Finally, I asked a question about the use of recommender systems as informational incentives for better reviews and Steven Tadelis discussed a study with monetary incentives for people to provide more informative reviews.
Session: Data for the Public Good
Speakers: Andra Ghent, Associate Professor of Finance, UNC; Matthew Linsday, Senior Principal, Mastercard; Skylar Olsen, Director of Economic Research, Zillow; Timothy Slaper, Co-Director, Indiana Business Research Center, Indiana University.
Moderator: Lucia Foster, Chief Economist, U.S. Census Bureau.
Talking points and key takeaways:
- Matthew Lindsay discussed various ways in which Mastercard can use the vast amount of information available to them to do good in society:
- Financial inclusion: From access to usage to security.
- Data driven insights to spur inclusive growth.
- Revitalizing economically distressed communities.
- Skylar Olsen discussed the value that Zillow has brought to the community by reducing informational asymmetries for people who want to buy a home by creating a database of 110 million homes. Zillow encourages researchers to use their data and they often try to build relationships with policymakers to maximize the social impact of their data. One example is their work on the relationship between rents and homelessness
Session: Putting Behavioral Economics into Practice
Speakers: Elizabeth Costa, Senior Director, The Behavioral Insights Team; Min Gong, Principal Behavioral Scientist / Data Scientist; Head of Applied Behavioral Science, Walmart; Emil Lantchev, Team Leader, Head of Portfolio Construction Research, Global Asset Association and Strategic Advisers, Fidelity Investments.
Moderator: Josh Scwartzstein, Associate Professor of Business Administration, Harvard Business School.
Talking points and key takeaways:
- Elizabeth Costa discussed their work on how do people make decisions which could be of interest to private platforms or the government. The BIT conduct both lab and field experiments all over the world. Another interesting question they are examining is how different aspects of choice architecture affect online behavior focusing mainly on consumer inattention and information overload. Finally she discussed some Fintech tools to help people manage spending. During the Q&A she discussed her pleasure in seeing the recognition of people that behavioral economics is not a collection of magic tricks but a systematic scientific reasoning.
- Min Gong mentioned that their moto is to try quickly – fail quickly – retry quickly and discussed about the planning fallacy – underestimation of resources needed to complete a project.
- Emil Lantchev discussed prospect theory and life-cycle investments. He mentioned that the reference point of prospect theory is people’s expectations of income at each given moment in time. He said that an important factor that is often overlooked is the uncertainty in future labor income which often covaries with asset returns. Finally he concluded by discussing how much asset customization is optimal.
Luncheon: The Ethical Algorithm
Speakers: Aaron Roth, 1940 Bicentennial Term Associate Professor of Computer and Information Science, University of Pennsylvania
Moderator: Anissa Tanweer, Research Scientist, Human-Centered Data Science, University of Washington.
Introduction: Peter Lorentzen, Associate Professor, University of San Fransisco
Talking points and key takeaways:
- This was a discussion of the area of fairness in Machine Learning (for more information see my related page here). Aaron discussed the literature generally as well as discussing his book “The Ethical Algorithm”, which I was lucky to get a free copy of in an earlier book signing session.
- Defining fairness, inclusion and discrimination is often much harder to do than one would think. What is more, the consequences of those definitions can be massive so we have to be very careful about it and start simple before complicating things. However, as more ML algorithms are being utilized for decision making, we have to make more progress soon.
- He discussed about the importance of interpretability of ML algorithms, emphasizing the need to know who are we interpreting them for.
- In the Q&A session he was asked whether shouldn’t we just introduce punishments for an algorithm designer for an algorithm which discriminates but he replied that that would disincentivize innovation and he reiterated that as of right now we do not have a clear understanding of what is a fair algorithm.
Session: Frontiers of Machine Learning and A.I.
Speakers: Zoubin Ghahramani, Chief Scientist, Uber
Introduction: Josh Wright, Chief Economist, iCIMS.
Talking points and key takeaways:
- Impressive AI advances in games which represent a very well-controlled world. Out in the real world there are still many challenges ahead.
- Zoubin mentioned that the overhype in AI which peaked around 2017 is now going back to where it should be. This is because over the past few years there has been a lot of trial and error of applying AI technologies in various contexts. This has resulted in both success and failure stories and we now have a much clearer picture of the promises and pitfalls of AI.
- He explained deep learning in simple terms as multi-layered neural networks representing non-linear functions that use some sort of ML penalized optimization method like gradient descent.
- Some limitations of deep learning are that it is very data hungry, computationally intensive (which contributes to global warming), uninterpretable black boxes and easily fooled by adversarial examples.
- Finally he concluded by saying that as AI becomes more embedded in important social decision making, it becomes more critical for systems to know what they don’t know (i.e. to be less confident about outliers)
Concluding session: Fireside Chat
Speaker: Jeff Wilkie, CEO, Amazon Worldwide Consumer.
Moderator: Steve Tadelis, Amazon Economist Fellow, AWS Business Systems / Professor of Economic, Business and Public Policy, Haas School of Business, UC Berkeley.
Talking points and key takeaways:
- This was perhaps one of the most interesting and captivating discussions between two leaders and innovators in the academia-tech world.
- Jeff Wilkie and Steve Tadelis talked about Wholefoods, Amazon Go among many other Amazon products.
- Jeff discussed Amazon.com’s efforts to raise the minimum wage, create amazing job satisfaction, job security and in general to make the company a pleasure to work in. Due to the size of the company, they hope that by doing so, others will follow too.
- As for the future of Economists in tech, Jeff said that he only expects it to grow, both within Amazon.com and in other companies who have been very interested in Economists over the past couple of years.
- They both emphasized the need to have respectful and thoughtful debates and about the importance of humility.
- Steve asked Jeff what keeps him up at night where Jeff talked about the impacts of what the company does in society. Due to its sheer size, any policy changes have massive societal impacts and for that reason, decision making is of critical importance.
A massive kudos to NABE, Amazon.com and Zillow as well as all the organizers of TEC2019 for a great conference but most importantly for promoting and advancing this incredible community. Can’t wait for the next one – TEC2020 in San Fransisco!
For more information check out NABE’s website.
