Maryam Mirzakhani (1977-2017)

A quick post to mark the passing of Iranian math genius Maryam Mirzakhani on July 15, 2017 at the age of 40 following a four year battle with cancer. The Iranian news website Press TV has an article and video you can find here. Her Wikipedia page is here. From the Press TV article: "Mirzakhani had recently been taken to hospital as her health condition worsened due to breast cancer. Cancerous cells had recently spread to her bone marrow. She had already been battling the disease for several years.

In 2014, Mirzakhani was awarded the coveted Fields Medal, also known as the Nobel Prize of mathematics. The 40-year-old, who used to teach at Stanford University, was also the first Iranian woman to be elected to the US National Academy of Sciences (NAS) in May 2016 in recognition of her “distinguished and continuing achievement in original research.”"

LA Times article notes her passing. You can find a video and story on her from WIRED, here.

Maryam Mirzakhan, rest in peace.

Here are some stories that caught my eye since the last post:

  • Quanta Magazine with a piece that starts "June Huh thought he had no talent for math until a chance meeting with a legendary mind. A decade later, his unorthodox approach to mathematical thinking has led to major breakthroughs."
  • Sputnik News on "A dean at Yale University has been put on leave after her sneering and sometimes racist comments on Yelp were exposed by students.". You'd think people would know by now: anything you post is out there forever.
  • Jacobin with a piece on Purdue University acquiring Kaplan University. Education is a business

Sagetex Basics: Factoring Quadratics

I've added another worksheet to the Sagetex Basics page on factoring quadratics. This worksheet creates 10 random quadratics on the first two pages. The line

while r1 == -r2:
r2 = Integer(randint(-10,10))

prevents any quadratic from being factorable by the difference of squares formula. If you want to allow quadratics of the form x^2-a^2 then comment those 2 lines out.

The variables outputP and outputA are the strings that contain the text for the Problems and for the output, respectively. This avoids having multiple sagesilent environments and allows a loop to take care of the repetition. The answers are printed out on the third page:

Here are some stories that caught my eye:

  • Oregon man fined $500 for using math to challenge red light cameras: "Mats Järlström learned this first-hand last year when the state of Oregon fined him $500 for publicly suggesting that yellow lights should last for slightly longer to accommodate cars making right turns....."
  • ZeroHedge with a post "Intellectual Intolerance - Stunning Speech From Stanford University Provost Exposes "The Threat From Within"" which says, "...But I’m actually more worried about the threat from within. Over the years, I have watched a growing intolerance at universities in this country – not intolerance along racial or ethnic or gender lines – there, we have made laudable progress. Rather, a kind of intellectual intolerance, a political one-sidedness, that is the antithesis of what universities should stand for. It manifests itself in many ways: in the intellectual monocultures that have taken over certain disciplines; in the demands to disinvite speakers and outlaw groups whose views we find offensive; in constant calls for the university itself to take political stands. We decry certain news outlets as echo chambers, while we fail to notice the echo chamber we’ve built around ourselves.""
  • Listverse has "10 Strange Facts About Pythagoras: Mathematician and Cult Leader"
  • Orrazz with a piece "There Are 5,000 Janitors in the U.S. with PhDs"
  • Shame on the AMS for hosting a racist piece on their website. The post "Get Out of the Way" drew enough comments to get the comments section closed. I suspect that had the comments been praise they'd have kept it open. The criticism isn't too surprising with an opening like this: "Not to alarm you, but I probably want you to quit your job, or at least take a demotion. Statistically speaking, you are probably taking up room that should go to someone else. If you are a white cis man (meaning you identify as male and you were assigned male at birth) you almost certainly should resign from your position of power. That’s right, please quit. Too difficult? Well, as a first step, at least get off your hiring committee, your curriculum committee, and make sure you’re replaced by a woman of color or trans person. Don’t have any in your department?  HOW SHOCKING....What can universities do? Well, that’s easier. Stop hiring white cis men (except as needed to get/retain people who are not white cis men) until the problem goes away. If you think this is a bad or un-serious idea, your sexism/racism/transphobia is showing. It is worth considering your complaints. Are you worried it’s unfair to men? Are you concerned the quality if your institution would plummet? Are you worried about all the brilliant minds you’d be missing? List your reasons and ask yourself which ones you’d value over your own freedom. Women are not free, and even our allies tend to only want us equal-ish. As long as we get there on their terms, as long as they lose nothing.If you are on a hiring committee, and you are looking at applicants and you see a stellar white male applicant, think long and hard about whether your department needs another white man. You are not hiring a researching robot who will output papers from a dark closet. You are hiring an educator, a role model, a spokesperson, an advisor, a committee person. When you hire a non-marginalized person, you are not just supporting this one applicant whom you like, you are rewarding a person who has been rewarded his whole life. You are justifying the system that makes his application look so good. You are not innocent. You are perpetuating a system that requires your participation if not your consent. When your female students of color have no role models in your department, that’s not “meritocracy”; that’s on you. Again, if you think the “great mathematicians” are disproportionately male because of meritocracy, then your sexism is showing...Not to alarm you, but statistically speaking you are the problem. Your very presence. I can’t tell you what is the best strategy for you to stop blocking my path. I can just ask that you please get out of my way." Absolutely disgusting. Could you imagine the furor and screams of racism if "white cis males" was replaced with some other ethnic group? Some of comments are interesting, in particular Math Colleague who notes "We are all painfully aware of the inequalities in faculty composition and trying hard to fix it. *Every* math department I know of is trying really hard to hire every qualified minority and female applicant out there (and by qualified I mean: a *very* generous ballpark within the hiring range of each department). The real problem is that there are not enough such candidates, and most departments end up making offers to the same few that are available in the market each year.". Math Colleague's explanation jibes with everything I saw in graduate school as well as out in the world where a woman and/or minority is highly sought after even when they are the weaker candidate. I know some minorities who were given a chance and were grateful for it.. Math Colleague sums up with "And all I can say is that if you did not get a tenure-track offer already, it is because you didn’t apply to enough places. If you what you want is a job at a top research institution, then you’ll need to prove that you are in the ‘generous ballpark’ of the research level you aspire to. I’ll be blunt: you have no grants, one publication, and the one paper is co-authored with your advisor. Write more papers, and we will hire you.You have great potential to make a difference for minorities and women in mathematics. Come up with a reasonable plan to help minorities, and NSF will fund it. But this post is unnecessarily destructive, discriminatory, inflammatory, and counterproductive to our inclusion efforts.". With a little more digging you'll find Piper on another blog where she says "They asked how many years spent on coursework (3), how many years spent on dissertation (10).  TEN YEARS?! WHAT KIND OF PERSON SPENDS TEN YEARS ON ONE DOCUMENT AND STILL HAS TO BUILD UP COURAGE TO DISCUSS IT OPENLY?... Oh, am I here because of Affirmative Action? Oh, am I here because I’m a woman? Oh, am I here because of a mistake? Okay, haha, I get it. Oh, do my friends think I took the spot of someone who deserved it more? That’s okay. It’s okay if my friends think that. Then there was the actual math. Funny story, I was exceptionally behind my peers when I got to Princeton. I’m tired of talking about it. I should have made myself a Tshirt that said “I only took 7 math classes before coming here, and my peers took 20 – 30, so that’s great.”". These comments appear to demonstrate she understands, at least to some degree, that she didn't get into Princeton for purely academic measurements-- she was "exceptionally behind" her peers. Princeton gets geniuses like Terry Tao if they want so there is lots of reason to suspect affirmative action has given her a chance. This, combined with 10 years work on a thesis which has led to one paper isn't awe inspiring. So Piper has gotten a chance to go to a prestigious university but that isn't enough. Remember the quote: "Stop hiring white cis men (except as needed to get/retain people who are not white cis men) until the problem goes away. If you think this is a bad or un-serious idea, your sexism/racism/transphobia is showing". Again, substitute any minority in for white cis male and ask yourself how that sounds. That would be grounds for someone losing their job; do you think it will happen to her? I don't. Or how about, "You are not hiring a researching robot who will output papers from a dark closet. You are hiring an educator, a role model, a spokesperson, an advisor, a committee person." Actually, the schools are trying to hire people qualified in math and producing research is part of the job description but regardless, white cis males are not incapable of being  a role models, spokespersons, advisors, and all the rest, are they? Contrast this Piper's situation with the piece above at Orazz with 5,000 PhDs working as janitors--I suspect they would be jealous of the chances she is getting. Piper's been given a chance that many lots of people haven't gotten and, invariably that means some have been the victims of reverse discrimination. Again, shame on the AMS post for hosting a piece that dismisses other people on the basis of being white males and white females, because "Remember having white cis women run the world is no kind of solution.". You can read a little more about Piper and her thesis by going to her blog.
  • The College Fix reports "After black student activists issued a demand list to American University in response to the racist-banana incident two weeks ago, the administration agreed to three demands.One of them is a ban on whites using a new “student lounge” for the rest of the spring semester."

Sage: Goldbach's Comet

I ran across Goldbach's Comet while surfing the net. This seemed like a straightforward Sage problem, so I spent some time creating a Sage Interact. You can find the code on the Python/Sage page. Copy and paste the code into a Sage cell server such as the ones on the Sage Sandbox page and you're ready to experiment. The image above was created by going up to 100,000. Though I didn't time it, I'd estimate it as taking about 30-40 minutes. The code posted has changed the max value to 1,000--you'll have to experiment what setting runs best on your computer. The link Goldbach.pdf in the bottom left creates a PDF file of your graph to download.

Here are some stories that caught my eye:

  • The fallout from the election has snowflakes melting: ZeroHedge with a piece on Ohio State offering "safe spaces" for students to survive the Trump inauguration. And do you remember how I pointed out how Trump was getting large crowds while Clinton was being chased around in Florida? Leave it to the media to explain that they didn't really miss the signs, they just chose not to mention them. But--if you believe them-- this means that the statisticians were even more clueless and inept. PJmedia with a piece "Chuck Todd Admits Media Treated Hillary with Kid Gloves" where"...NBC’s Chuck Todd confessed that he and others in the mainstream news media played down just how despised Hillary Clinton was in the heartland due to the fear of appearing “sexist.” ...“Where I think political correctness got in the way of what we all knew as reporters and didn’t fully deliver was how hated the Clintons were in the heartland,” the “Meet the Press” host admitted Thursday to former Bush White House press secretary Ari Fleischer in a interview for the “1947” podcast. “And I think it was a fear of, ‘Oh, is it going to look like it’s sexist, anti-woman if we say that?’” he added, pointing out that on the hustings he saw numerous “Hillary for Prison” signs adorning the front yards of rural America. “I think we underplayed it a little bit out of political correctness fears,” Mr. Todd said. “No member of the press corps wants to look like they’re singling out a group and making a group feel bad, right, whatever that [group] is. “If we sort of were straight-up honest and blunt about hey do we understand the level of hatred that’s out there and you know, all the Hillary for Prison signs that are out there, we certainly would have at least made the viewer know, hey, you know, she’s not well-liked in some places in this country in ways that’s times 10 when it comes to Trump,” he said." Refreshing honesty in reporting---once it's too late to do you any good. They aren't called presstitutes for nothing.
  • Anonymous with 10 lies you were taught in high school.
  • RT reports "Scientists add letters to DNA alphabet to create ‘semisynthetic’ life"
  • Tolerant snowflakes, ZeroHedge reports, "The NYU College Republicans likely had no idea what they were setting themselves up for when they invited Vice Media co-founder Gavin McInnes to speak on their campus.  Two years ago such an event almost certainly would have gone completely unnoticed but for the the couple of hundred students in attendance.  But in today's post-Trumpian world, every appearance of a conservative personality is taken by intolerant Leftists as just another opportunity to destroy public/private property, incite violence and shout catchy slogans incorporating the word "Nazi." According to Reuters, McInnes' speech was cut short when protesters rushed into the room where he was speaking and began interrupting him.  The protesters subsequently scuffled with police officers and McInnes supporters outside the university's student center in New York City, where he was invited to speak by NYU College Republicans."" And check out the screaming woman, claiming to be a profession, with a tirade of profanity. Because free speech only matters to these goons when it is speech they agree with.
  • Where else than California for a 1.5 billion dollar budget error. "...we, as taxpayers, generally rely on our expensive budget office employees to at least present annual budgets that reflect sound mathematics and accounting principles.  Unfortunately, that seems to be too much to ask of the math-challenged administration of California Governor Jerry Brown which decided to double count certain cost savings and simply "forgot" to incorporate other expenses altogether....Embarrassingly, when asked about the "mistakes" that resulted in a $1.6 billion budget deficit, the Chief Deputy Director of Brown's Department of Finance could offer no other explanation than that the "math was wrong" while another spokesman admitted, “There’s no other way to describe this other than a straight up error in accounting, which we deeply regret."" Don't hold your breath waiting for accountability even though that would get you fired at your job. Is a tax increase coming?
  • Intellihub with the statistic of the day, "Here is a staggering statistic from the National Alliance on Mental Illness (NAMI): “More than 25 percent of college students have been diagnosed or treated by a professional for a mental health condition within the past year.Let that sink in. 25 percent. Colleges are basically clinics. Psychiatric centers. Colleges have been taken over. A soft coup has occurred, out of view.You want to know where all this victim-oriented “I’m triggered” and “I need a safe space” comes from? You just found it. It’s a short step from being diagnosed with a mental disorder to adopting the role of being super-sensitive to “triggers.” You could call it a self-fulfilling prophecy. “If I have a mental disorder, then I’m a victim, and then what people say and do around me is going disturb me…and I’ll prove it.”
  • HeatStreet with a piece on "bias response teams" which notes, "....“Bias Response Teams” are creeping onto university campuses across the country. This was the conclusion of the first national survey of Bias Response Teams done by the Foundation for Individual Rights in Education (FIRE). The report identified 232 public and private American colleges and universities that had bias response teams on their campuses in 2016, affecting around 2.8 million students. BRTs encourage students to formally report on other students and faculty members whenever they perceive that someone’s speech is “biased,” which threatens free speech. Most universities receive a variety of complaints from students, including students who encounter “offensive” yet legally protected speech, but rather than responding to these incidents fairly if there’s an actual threat, campuses with Bias Response Teams conduct an investigation and if the “respondent” is found “guilty”, invite them for a “hearing”. Examples of Bias Response Teams exercising their power include a student humor publication at the University of California, San Diego (UCSD) that lost its funding after making fun of “safe spaces” on campus. They got their funding pulled after people filed bias incident reports, one openly calling the university to “stop funding”. At Ohio State University, students had to attend a “mandatory floor meeting about triggering events” after a group of students was reported for sharing memes comparing Hillary Clinton to Adolf Hitler. Students who dressed up as the “Three Blind Mice” for Halloween at the University of Wisconsin at Platteville were also reported, as the complainant felt the costumes could be interpreted as making of fun of disabilities."
  • Can a cone roll uphill? Yes!
  • Unz Review with an interesting educational story: "No Thug Left Behind"
  • Alphago was playing incognito over the internet and destroying the competition, "...AlphaGo took a clandestine route to toppling the worl'd top ranking players. Starting on December 29th, 2016, a newcomer used the account names Magister and Master on the Tygem and FoxGo servers and caused quite a stir among players and observers alike. Master defeated a long list of top Go players including Korea’s Park Jung-hwan, rated No. 3 in the world, Japan’s Iyama Yuta (No. 5) and China's Ke Jie who has held the No. 1 spot since mid 2014....Master won successive 51 games before his 52nd rival, Chen Yaoye (No.13), went offline, forcing the game to be recorded as a tie and by January 4th had clocked up 60 wins, that single draw and no losses....the user registered as "Master" provided his real identity - AlphaGo's Doctor Huang, a member of the Google DeepMind team - on foxwq.com.While Ke Jie still has his No. 1 ranking AlphaGo is already No. 2 and it seems doubtful that any human will ever beat it again. This is something that Ke Jie is finding hard to accept. Ali Jabarin, a professional Go player, who came across Ke soon after one of his defeats by the AI reported: "He [was] a bit shocked... just repeating 'it's too strong.'"". Now GBTimes reports AlphaGo will play the worlds top human in April. "The match between Ke Jie and AlphaGo will take place in Wuzhen, China’s eastern province of Zhejiang. If Ke loses the best-of-three match, China, Japan and South Korea may each form a team and challenge AlphaGo."

SageTex: Relabeling the Vertices

If you use Sage to generate graphs you'll find there are times when the vertices aren't labelled the way you expect them or want them to be:

In this case the graph is the line graph of a complete graph on 5 vertices, so labelling the vertices with 2-tuples makes sense--the vertex (0,2) would correspond to the edge (in K_5) from vertex 0 to vertex 2. but the "None" part of each label is annoying, to say the least. And that has created an additional problem of vertices which are too big. Luckily Sage has the ability to relabel vertices. In this case, I'm going to relabel using the vertices as just 2-tuples corresponding to vertices in K_5 of {1,2,3,4,5} to get:

The relevant part of the code is

g = graphs.CompleteGraph(m)
H= g.line_graph().complement()
H.relabel([(i,j) for i in range(1,m+1) for j in range(i+1,m+1)])

The first line creates the complete graph on m, equal to 5, vertices. The second line uses Sage's knowledge of graphs to create the line graph of K_5. The line

H.relabel([(i,j) for i in range(1,m+1) for j in range(i+1,m+1)])

generates the desired labels and as a result the vertices are not so huge. The code for the figure above can be found on the Graph Theory, Sage, LaTeX page.

Here are some stories that caught my eye.

  • A new math series has launched on PBS: NOVA calls it Inifinte Secrets. And yes, the woman waves her hands a lot.
  • President elect Donald Trump has picked Betsy DeVos as Education Secretary. She is anti Common Core. KHOU has 5 things you should know about her.
  • Gallup pats themselves on the back for accurately predicting the results of the presidential election. They've also given some excuses to explain the failure at the local level: "State polls typically have smaller sample sizes, have more variable quality depending on what organization conducts the poll, are estimating an outcome that can shift more readily because the population is smaller, are often conducted further away from Election Day and are more dependent on precision in estimates of turnout by geography...To the degree that organizations want to predict the Electoral College, they are going to have to find ways to finance or encourage larger-sample, higher-quality state polls, rather than relying on the haphazard polls that happen to be conducted in the various states..". So an "F" performance by most everyone's count but their's. And notice the certainty of the predictions were based on "haphazard polls"---never heard that mentioned before they made themselves look like fools.
  • CNBC reports that a partner of ETS, the company that provides security for the SAT, has helped to compromise the SAT: "When the new SAT was given for the first time in March, the owner of the test took unprecedented steps to stop "bad actors" from collecting and circulating material from the all-important college entrance exam. But in the months since, China's largest private education company has been subverting efforts to prevent cheating, Reuters found. The company, New Oriental Education & Technology Group Inc, has regularly provided items from the tests to clients shortly after the exams are administered. Because material from past SATs is typically reused on later exams, the items New Oriental is distributing could provide test-takers with an unfair advantage....Hundreds of thousands of students enroll in New Oriental's test-prep classes. It has a stock market capitalization of $6.6 billion and a listing on the New York Stock Exchange. New Oriental's founder and executive chairman, Michael Minhong Yu, is a business celebrity in China. Yu's company is also a business partner of Educational Testing Service, or ETS - the New Jersey non-profit that both owns the TOEFL and provides security for the SAT.
  • Reuters on New Oriental hooking up with colleges, too: "Eight former and current New Oriental employees and 17 former Dipont employees told Reuters the firms have engaged in college application fraud, including writing application essays and teacher recommendations, and falsifying high school transcripts. The New Oriental employees said most clients lacked the language skills to write their own essays or personal statements, so counselors wrote them; only the top students did original work. New Oriental and Dipont deny condoning or wittingly engaging in application fraud.".
  • Teachers this generation have a different set of challenges. Sott.net has a piece on "Police believe students may have hatched an "elaborate plot" to frame their teacher for viewing pornography on his computer at school. Investigators cleared a teacher at Longfellow Middle and High School of wrongdoing after students accused him in November of watching a porn video on his school computer in class, Fox 59 reports. "Investigators believe the teacher left a laptop sitting out, and students were able to breach the (Indianapolis Public Schools') firewall and load a porn website onto the computer," according to the news site. "Police say they discovered inconsistencies in the allegation against the teacher," WTHR reports. "They say the students 'may have been involved in an elaborate plot to frame the teacher.'" "
  • RT has the most gripping video footage, courtesy BBC’s ‘Planet Earth 2’ of a lizard being chased by snakes---lots and lots and lots of snakes. This is a video you must see!!
  • RT on teachers getting in trouble for anti-Trump comments. Meanwhile, Reason.com has a piece on all the hate crimes inspired by Trump that turned out to be hoaxes. And college snowflakes are melting down again because of Donald Trump winning the election "The University of Michigan’s distressed students were provided with Play-Doh and coloring books, as they sought comfort and distraction. A University of Michigan professor postponed an exam after many students complained about their “serious stress” over the election results. Cornell University held a campuswide “cry-in,” with officials handing out tissues and hot chocolate. One Cornell student said, “I’m looking into flights back to Bangladesh right now so I can remove myself before Trump repatriates me.” The College Fix reported that “a dorm at the University of Pennsylvania … hosted a post-election ‘Breathing Space’ for students stressed out by election results that included cuddling with cats and a puppy, coloring and crafting, and snacks such as tea and chocolate. The University of Kansas reminded its stressed-out students that therapy dogs, a regular campus feature, were available. An economics professor at Yale University made his midterm exam “optional” in response to “many heartfelt notes from students who are in shock over the election returns.” At Columbia University and its sister college, Barnard, students petitioned their professors to cancel classes and postpone exams because they were fearful for their lives and they couldn’t take an exam while crying....Does a person even belong in college if he cannot handle or tolerate differing opinions? My answer is no.” Unfortunately, that's the coming generation. Wonder how they'll handle the inauguration?!? (spoiler alert: badly)
  • Wearechange.org on "Students at the University of Pennsylvania have decided that William Shakespeare “doesn’t represent a diverse range of writers,” so they ripped his portrait off the wall in the English department. The triggered students replaced the famous playwrights portrait with that of author Audre Lorde, a black lesbian feminist and civil rights activist who is best known for her poetry. While she is absolutely deserving of honor and respect, there are likely plenty of walls for both. The school, instead of punishing the students for vandalism, are rewarding them and caving to their petulant demands.....Campus Reform noted that the school’s code of conduct expressly prohibits students from  “stealing, damaging, defacing, or misusing the property or facilities of the university or of others.” Yet, universities across the nation continue to teach their students, who are primarily adults, that tantrums work. Sadly, they will likely be ill-prepared to deal with the real world where everything isn’t sunshine, rainbows, and getting their way."
  • The challenges of teaching today's darlings: RT has a headline which says it all. "Master bakers: Omaha high-school pranksters trick teacher into swallowing semen frosting". From the article, "Three freshmen thought it would be funny to masturbate into a container and add the fluid to the frosting ingredients, the Omaha World Herald reports.The students’ 59-year-old female teacher then suffered the misfortune of tasting the boys’ finished product. She noticed the turnovers tasted a little funky.Another student later told the teacher that he had heard the culprits talking about their plans to add something salty to the ingredients.The three boys, ages 14 and 15, were questioned by school authorities. Two admitted to their deed, and a third said he had chickened out of the act.Police were called and the boys’ frosting containers seized.“The students will face consequences,” said Brandi Petersen, a spokesperson for Westside High. “We do not tolerate anything of this nature.Sadly, there is no law against adding bodily fluids to food. While the frosting was no doubt an assault on the senses, it doesn’t count as an actual assault as there was no bodily injury."
  • 21st Century Wire hosts a piece asserting "disturbing development at the University of Oregon, whose administration made clear to its faculty last week that if you say things about race, sexual orientation, sex, religion and so on that enough people find offensive, you could get suspended and possibly even fired. This can happen even to tenured faculty members or to anyone else."

Trump. Because statistics isn't math

Let me just start this touchy subject by saying I didn't vote in the election, which was the winning choice this election cycle, because I didn't like any candidate. George Carlin says it best. That said, the coverage of the election and more importantly the polls were obviously biased/rigged/wrong, pick the word you like best. And given the absolute shock by soooooo many people, I have to revisit the truth, as I mentioned just last post, that statistics of polling isn't math. That showed that not only do the +/- percent errors not mean what you think, but trained hacks statisticians can have exactly the same data and come up with several different answer. That's not how it's done in math. Previously we saw the BS field of statistics make a major error with respect to Brexit, when virtually everybody got it wrong. Trillions of market cap were wiped off the financial markets. That great job was followed up with the recent US election. Once again statisticians were wrong-- BADLY wrong --with over a trillion in losses for the week. Check out Goldseek's summary of polling data before the election. TheWrap also has plethora of pre-election predictions for electoral college, percentage chance of winning, and percentage to win by: L.A. Times: Clinton 352, Trump 186, Moody’s Analytics: Clinton 332, Trump 206, Rothenberg & Gonzales: Clinton 323, Trump 197, Sabato: Clinton 322, Trump 216, FiveThirtyEight: Clinton 302, Trump 235, Associated Press: Clinton 274, Trump 190, New York Times Clinton 85 percent chance of winning, Hypermind: Clinton 74 percent chance of winning, PredictWise: Clinton 89 percent chance of winning, Reuters/Ipsos States of the Nation: Clinton 90 percent chance of winning, ABC News Political Analyst Matthew Dowd: Clinton 95 percent chance of winning, Bloomberg: Clinton +3, CBS News: Clinton +4, Fox News: Clinton +4,Reuters: Clinton +3, ABC/Washington Post: Clinton +4, Monmouth: Clinton +6, Economist/YouGov: Clinton +4, Rasmussem: Clinton +2, NBC News: Clinton +6.

Those numbers were backed by strident rhetoric regarding the impending, decisive victory of  Clinton. NYPost on the pundits who got it so wrong, including: "Deadspin columnist and GQ correspondent Drew Magary went so far as to publish a scathing piece about the Republican candidate on Sunday — titledDonald Trump Is Going To Get His Ass Kicked On Tuesday.”" TheNewser mentions "Princeton ace Sam Wang, who in mid-October tweeted, "It is totally over. If Trump wins more than 240 electoral votes, I will eat a bug."" . Let's hope he is a man of his word and does it. But don't hold your breath. Even some of the margins of victory were way off: The Telegraph reports "In Iowa, the result was over six percentage points more in favour of Trump than the final polling average. In Ohio - the key bellwether state that has sided with the winning candidate in every presidential election since 1964 - the average polling was five points out....Non-college educated white people make up 41.2 per cent of Indiana's population, for example. The polling gap here was 8.6 percentage points - with the polls giving an average 10.7 point lead to Trump, compared to the actual 19.3 point lead he managed...Wisconsin...was a huge 7.4 percentage point gap between the polls and the result here: the average polling on Real Clear Politics before the vote had Clinton winning by 6.5 percentage points, when in fact Trump claimed the state by 0.9 points."

Which led TheWrap to ask how is it possible to be so wrong? While statisticians are confident of their pre-election prediction, once their wrong, they have no trouble coming up with a excuse reason as to why. From the link "Another theory is that many Trump supporters were simply ashamed to admit that they were going to vote for the GOP nominee and lied to pollsters. Trump is one of the most polarizing figures in American history and many voters wouldn’t even admit it to a complete stranger looking to conduct a poll.". That was a Brexit excuse, too. And yet they were still so sure of their prediction, rather than reduce their percentage of winning to reflect the possible problem. Maybe next time? Some claim the results are lies. Maybe some, but with trillions of dollars of investment capital being lost through Brexit and the US election, there is a lot of "smart money" that should have known better. When you mess things up that badly in math, you look through your work, find the error and fix the problem. In statistics, you find an excuse, or better yet multiple excuses as to why you're wrong and do it again. With each reason you hear ask yourself "then how can they be so sure?".

Of course, some will state "nobody saw it coming". But that's true only if you watch MSM. Check out the Keiser Report, for example where at the 9:35 mark they mention Chris Whalen and Jim Rickards. Chris Whalen is interviewed there. Jim Rickards had a more guarded prediction here. But nobody was more vocal than Andy Hoffman. And, oh yeah, he predicted Brexit, too. But these guy aren't pollsters. Curious, There's a lesson in there somewhere.

And it's not like it was difficult to see some problems: Kaine canceling a Sarasota, Florida rally after only 30 people (about half of which were press) showed up at his West Palm Beach rally. Meanwhile Trump had a different experience: "Republican presidential nominee Donald Trump held a massive rally Tuesday evening at an airplane hangar in Melbourne, Florida to a massive crowd of about 10,000 supporters-while an equal number were turned away when the fire marshal declared the event at capacity....Neal interviewed the last person in line allowed in, Rachel Padrick-Miller. She had been in since 2 p.m. When the fire marshal cut off the line she said ‘thousands were trying to break through the line…jumping the barricades.". And Clinton? "The New York Times features a photo of Hillary Clinton being welcomed to an early voting site in Pompano Beach, Florida on Sunday…
Surrounded by screaming TRUMP supporters!
". Not to mention people chanting "Lock her up!". And see the 11:33 mark of the Keiser Report link above where they witnessed so much more Trump support in North Carolina that they couldn't reconcile with what the pollsters were predicting guessing.  This attendance deficit was in lots of states which witnessed packed Trump events versus a few hundred Clinton supporters. Statistical hacks saw nothing wrong with there models despite the fact that these are the people more likely to go out and vote [See Reuters story below]. They were also aware of Wikileaks and the other umpteen excuses and yet they still were sure of a Clinton win. Now they will "fix" the models. But don't expect the predictions to get any better 10 years from now--statistics isn't math.

Here are some stories that caught my eye:

  • The World Chess Championship has begun. Magnus Carlsen, the champion, takes on Sergey Kajarkin. The official site is here, but I'd recommend Chessbomb for following the games (unless you're willing to pay up for coverage).
  • The best chess is being played between the computers. Komodo has already been dethroned. This year, Stockfish battles Houdini for the 9th TCEC Superfinal. Follow it here.
  • Sott.net reports "Professors across the nation are cancelling classes Wednesday to help students deal with their emotional distress as they come to grips with the "shocking" election of Donald Trump. ". Poor, poor snowflake.
  • The Atlantic has a more in depth look of the various excuses reasons why pollsters were so wrong.
  • TruePundit on Newseek's election goof: "The partisan hacks at Newsweek are recalling 125,000 copies of its Hillary Clinton “Madam President” issue. The brutal mistake, printing and shipping the issues to wholesalers and retailers weeks before the election, is estimated to cost Newsweek a loss of approximately $500,000. But Newsweek is not new to making lousy management and fiscal decisions. Newsweek claimed it made a Donald Trump edition too although it never shipped it or produced a verified copy of it. Now Newsweek says it will print and ship the Trump edition and hope to break even on the misadventure."
  • ZeroHedge on fudged polls: "The problem, said Cliff Young, president of Ipsos Public Affairs US, the polling partner of Reuters, came down to the models the pollsters used to predict who would vote - the so-called likely voters. The models almost universally miscalculated how turnout was distributed among different demographic groups, Young said. And turnout was lower than expected, a result that generally favors Republican candidates. Ultimately, missing that shift in the state polls tripped up the predictions. It also highlights how the otherwise empirical process of polling rests on a subjective foundation. Each pollster must make a decision about turnout. Their decisions are informed by historical voting patterns. But the actual turnout in each state is unknowable before election day."

 

Polling: statistics isn't really math

I've said it before: statistics isn't really math. It's an application of math like engineering,  mathematical economics, mathematical finance, predicting the weather, etc that uses math to claim respectability. Sure, theoretical statistics is math--akin to analysis--but actually applying statistics to numerous problems when there is no certainty that underlying assumptions (such as the data being random) are true is just wrong. In a previous post, "Statistics Isn't Really Math" I looked at some of the problems with statistics. In particular, I cited the post at AMSTAT News, (The Membership Magazine of the American Statistical Association) saying  "Statistics, however, is not a subfield of mathematics. Like economics and physics, statistics uses mathematics in essential ways, “but has origins, subject matter, foundational questions, and standards that are distinct from those of mathematics” (Moore, 1988, p. 3). David Moore, statistics educator and former president of the American Statistical Association, gives the following four compelling reasons why statistics is a separate discipline from mathematics:

  • Statistics does not originate within mathematics
  • The aims and foundational controversies of statistics are unrelated to those of mathematics
  • The standards of excellence in statistics differ from those of mathematics
  • Statistics does not participate in the inter-relationships among subfields that characterize contemporary mathematics

Statistics exists because of the need for other disciplines to examine and explain variation in their data."

That's a nice, clean authoritative explanation by statisticians as to why statistics isn't really math.

I followed that post up later with "Brexit: because statistics isn't really math" when virtually all the hacks statisticians made a horrendous call on Brexit. The bad prediction wiped out trillions in market cap when the vote went against what almost every public poll believed would happen.

Now with the US election coming up, I figure I should highlight a couple of new pieces that caught my attention. The first is a link that was brought to my attention by a reader, courtesy of the NY Times: "When you hear the margin of error is plus or minus 3 percent, think 7 instead". Now margin of error is a key term in polling statistics; except, according to this article, prepare to have your knowledge challenged: "In a new paper with Andrew Gelman and Houshmand Shirani-Mehr, we examined 4,221 late-campaign polls — every public poll we could find — for 608 state-level presidential, Senate and governor’s races between 1998 and 2014. Comparing those polls’ results with actual electoral results, we find the historical margin of error is plus or minus six to seven percentage points. (Yes, that’s an error range of 12 to 14 points, not the typically reported 6 or 7.)". Yes, throw out everything you learned about confidence intervals. But wait, it gets worse. A link off of that article takes you to here, an article entitled: "We Gave Four Good Pollsters the Same Raw Data. They Had Four Different Results" which is about just what the title indicates. Four pollsters were given exactly the same set of polling data and 4 different predictions. This is NOT what you'd get in a math course. The reason?: "Polling results rely as much on the judgments of pollsters as on the science of survey methodology. Two good pollsters, both looking at the same underlying data, could come up with two very different results.How so? Because pollsters make a series of decisions when designing their survey, from determining likely voters to adjusting their respondents to match the demographics of the electorate. These decisions are hard. They usually take place behind the scenes, and they can make a huge difference....Pollsters usually make statistical adjustments to make sure that their sample represents the population – in this case, voters in Florida. They usually do so by giving more weight to respondents from underrepresented groups.". Got that? Pollsters tamper with adjust the data as they feel like.

In fact, ZeroHedge has this post where polls contradict the margin error of others, and another post looking at the methodology behind a recent Washington Post poll: "Of course, like many of the recent polls from the likes of Reuters, ABC and The Washington Post, something curious emerges when you look just beneath the surface of the headline 12-point lead."METHODOLOGY – This ABC News poll was conducted by landline and cellular telephone Oct. 20-22, 2016, in English and Spanish, among a random national sample of 874 likely voters. Results have a margin of sampling error of 3.5 points, including the design effect. Partisan divisions are 36-27-31 percent, Democrats - Republicans - Independents."As we've pointed out numerous times in the past, in response to Reuters' efforts to "tweak" their polls, per the The Pew Research Center, at least since 1992, democrats have never enjoyed a 9-point registration gap despite the folks at ABC and The Washington Post somehow convincing themselves it was a reasonable margin."

Finally, there is this somewhat humorous post "Here's The 30 Seconds After The Last Debate That CNN Would Rather You Didn't See" where CNN polling has a 52% to 39% win for Clinton in the 3rd debate "So when the CNN focus group was asked "did this debate help anyone make up their mind or possibly change their vote", the results did not turn out how Goebbels they expected...

  • 5 Clinton
  • 10 Trump
  • 0 3rd Party
  • 6 Undecided

A much, much different result than there poll. Polling data is not the same coin flip data and the situation is even worse with respect to the integrity of the data---Stanford University called attention to Election Fraud here. By now it should be even more obvious that statistics isn't really math: the margin of error doesn't mean what it should and qualified statisticians with exactly the same data come up with different answers.

Here are some events that caught my eye lately.

  • Poor Nigel Short can't really catch a break. After getting into trouble with PC police for his comments on women which got twisted and blown way out of proportion, he had a 6 game match with Hou Yifan, the highest rated woman chess player in the world. The match was actually less close than the score would indicate with Short winning the match after 5 games in which he was never really in trouble--before losing badly in the final game. Was it a gift?--after all, he'd won the match, the last game wouldn't be rated, and it would be good gesture. Whatever the reason for the one game in which Short played badly, he got punished yet again. Chess.com has a report which states, "Well, Short had secured match victory after the fifth game, and later that day, he discovered that according to official regulations the last game should not be rated. Paragraph 6.5 of the FIDE Rating Regulations says:"Where a match is over a specific number of games, those played after one player has won shall not be rated." Short had an email discussion with tournament director Loek van Wely late Friday night. Van Wely wasn't immediately convinced. In fact, two years ago, when Anish Giri had won his match before the last game with Alexey Shirov, that sixth game was rated." So Short showed clear dominance throughout the first 5 games, never being in any danger, knew the last game wasn't going to be rated before it was played and "finished" the match with a bad loss, only to find the game WAS rated, which violated FIDE rules. Now he's not happy. And check out the footnote like reference to Short winning the match at Chessbase. Had he lost there would have been a BIG story on woman beats man in grudge match.
  • American Thinker with a piece on  precious snowflakes scared of Halloween: "College offers round-the-clock counseling for students 'troubled' by Halloween costumes"
  • EAGNews on the high school principle who told a student to remove his headphones in school, and  "When the student refused, Tossman attempted to remove the headphones, which allegedly sent Penzo into a rage. “ … (T)he 18-year-old student cold-cocked the principal,” according to the news site. “Penzo continued to pounce on Tossman, socking him several times in the face, causing swelling and lacerations around both the principal’s eyes.”A prepared statement released by the school contends “The NYPD immediately responded” and took Penzo into custody.""
  • FiveThirtyEight reports "A new study shows that first-grade teachers consistently rate girls’ math ability below boys’ — even when they have the same achievement level and learning style. The study out today in the journal AERA Open from researchers at New York University and the University of Illinois at Urbana-Champaign seems to represent a setback for gender equity in math. A widely reported 2008 study found that girls score as well as boys do on standardized state math tests. But the latest study suggests that early in their math education, many girls run into a teacher who perceives them as being worse at the subject than they are — which could discourage some of them from heading down a path that could lead to a career in math, science or engineering.". This is surprising to me because I have the impression that most math teachers at that level are women. Unfortunately I don't have the data to back that up. Is anyone aware of the data for this?
  • A powerful image of education winds up in a story on Sott.net.  Take a look at the school in Afghanistan halfway down the page. I've got to believe that some teachers would find that useful in their class.
  • Geekwire with a piece "Meet the minds behind Axiomatic: An art project based in theoretical mathematics"
  • Carlsen versus Nakamura in a blitz match today. Chess.com tells you how to watch it online.

Irrational Numbers with Patterns

img_20160919_163743715I'm not really posting anymore but I had to make an exception. The bad quality picture above is from a student's math class and the worksheet says: "Decimals that never end and have no pattern, think MUMBO JUMBO NUMBERS" in describing irrational numbers. Some context: the student had prealgebra last year where they learned that rational numbers were numbers that can be written as an integer divided by a nonzero integer and that irrational numbers were real numbers that weren't rational. More importantly, the student had learned the decimal characterization of rational and irrational numbers: rational numbers are terminating or nonrepeating and irrational numbers are nonterminating and nonrepeating.

So when the student, now in algebra, went through the beginning of the school year review of the real number system with this worksheet you have to cringe. First, the algebra class has learned the important definitions for rational and irrational numbers, so why not review those terms? That SHOULD be the goal. Second, the characterization of irrational numbers as "have no pattern" is wrong and she uses pi as an the only example to back up her claim. The teacher has given the class a worksheet with wrong information and doesn't know it. Welcome to a quality public school.

So let's give some quick examples to of irrational numbers which have decimals that never end BUT have a pattern.

.123456789101112131415......

Yes, ... is ambiguous, so let's describe the pattern. Just count the positive integers and put them together after the decimal. Understanding the pattern you know the next digits are ....161718192021..... and you can calculate whatever place after the decimal you want if you need to because there is a pattern. Here's another example:

.101101110111101111101111110.....

The pattern is to have separate one "1" from two "1"s from three "1"s from...using a 0. Under that pattern, you know the next digits are 11111110111111110..... and so on. Both of these decimal numbers are nonterminating and nonrepeating. You can't shouldn't make up your own math, especially when it's wrong. It causes unnecessary confusion.

Here are some stories that caught my eye recently:

  • From the Daily Mail, "Meet the boy geniuses who developed a math theorem that calculates problems faster than a COMPUTER - despite still being in high school"
  • Russian chess announces the passing of chess trainer Mark Dvoretsky. As someone fortunate enough to have attended some of his training sessions, I appreciate just what a tragic loss this is for chess.
  • Poor Chicago teachers! Well, not poor exactly. ZeroHedge reports they are ready to strike yet again. "Chicago teachers have a 13% raise (over four years) offer on the table, but that is not enough. They set a strike date of October 11 because the city wants the union to contribute more than 2% for their underfunded pensions, among the worst funded pensions in the nation. The Chicago public school system is bankrupt. Its bonds are deep in junk status....an affiliate of the American Federation of Teachers, is gearing up for a strike. It would be the union’s second in four years, despite the fact that the median salary for a teacher in Chicago is $78,169. When you add another $27,564 for various benefits, the total compensation for a teacher – good, bad or middling – becomes almost $106K per annum. (Please keep in mind teachers work 180 days a year, while employees in other professions typically work for 240 to 250 days.) In retirement, the average Chicago teacher receives a hefty $50,000 a year.
  • Truthdig with a sobering look at the status of adjunct teachers. With the hurdles to teach at public high school too onerous they are the working poor.
  • Sott.net with a well covered story on a Michigan student who, "...noticed nasty discolored water coming from the sink in her school, so she took a picture and shared it with social media. Hazel Juco was hoping that she could raise awareness about the potentially dangerous issue, and perhaps get the problem fixed. Unfortunately, the school was not impressed with the student's whistleblowing and actually suspended her for violating the school's policy of taking photos in the bathroom."
  • Propublica on "Education Department Terminates Agency That Allowed Predatory For-profit Colleges to Thrive"
  • Daily Mail with a viral image. You can't trust your senses.
  • Remember the poor Yale professor who was screamed at by hysterical brats? The Federalist has more video of what led up to the incident. "Altogether, the footage is nearly 25 minutes long — during which Nicholas manages to keep his cool in an attempt to rationally discuss his wife’s email with a cluster of impassioned students. The scene is chaotic — students verbally attack Nicholas, demanding he apologize for his wife’s “racist” comments. Ultimately, he does not decry the content of Erika’s email, but he emphatically and repeatedly apologizes for any pain her words caused....Based on this footage, it’s clear the students at Yale were incapable of having a rational discussion with another individual who dared to push back against their hasty accusations of racism. Despite what others have tried to claim, it’s plain as day many of those students who loudly disrupted the campus with their protest really were crybabies that threw tizzies over an email about Halloween costumes."
  • Lew Rockwell with a nice video on the value of a college degree.
  • American Thinker has some thoughts on reforming higher education: "We need to increase by 40% over the next ten years the number of people with college degrees, a state official once lectured us. Sitting in the audience, I thought, we know how to give people degrees. We no longer know how to give them a college education...."
  • Quartz with a piece on "Seven Rhode Island universities, including Brown and Rhode Island College, will move to open-license textbooks in a bid to save students $5 million over the next five years, the governor announced Tuesday"
  • The Federalist spotlights just how worthless a PhD in education is in reporting on  a PhD candidate who claims "Science Is Sexist Because It’s Not Subjective". "Throughout her dissertation, Parson assumes and asserts that women and minorities are uniquely challenged by the idea that science can provide objective information about the natural world. This is an unfair assumption, she says, because the concept of objectivity is too hard for women and minorities to understand."

 

Odds and Ends: Aug 30, 2016

Just a quick post. As I'm no longer teaching I won't be working on the website as much.

Here are a few stories that caught my eye since the last update. A lot of links for you science teachers:

  • RT with an entertaining video from a finalist in the Best Illusion of the Year Contest 2016. Science teachers, take note!
  • Listverse has "10 Reasons Academic Journals Are Filled With Junk Science". Number 9 is a good one, related to bogus journals: "A pair of computer scientists, frustrated with unwanted spam from a pay-for-publication journal, submitted a joke article. It was 10 pages of the same seven words repeated over and over: “Get me off your f—king mailing list.”"
  • Reason adds its voice on junk science, "Sarewitz cites several examples of bad science that I reported in my February article "Broken Science." These include a major biotech company's finding in 2012 that only six out of 53 landmark published preclinical cancer studies could be replicated. Researchers at a leading pharmaceutical company reported that they could not replicate 43 of the 67 published preclinical studies that the company had been relying on to develop cancer and cardiovascular treatments and diagnostics. In 2015, only about a third of 100 psychological studies published in three leading psychology journals could be adequately replicated.....Sarewitz also notes that 1,000 peer-reviewed and published breast cancer research studies turned out to be using a skin cancer cell line instead. Furthermore, when amyotrophic lateral sclerosis researchers tested more than 100 potential drugs reported to slow disease progression in mouse models, none were found to be beneficial when tested on the same mouse strains. A 2016 article suggested that fMRI brain imaging studies suffered from a 70 percent false positive rate. Sarewitz also notes that decades of nutritional dogma about the alleged health dangers of salt, fats, and red meat appears to be wrong."
  • Signs of the Times with "Madam Marie Curie's research papers still radioactive 100+ years later".
  • Signs of the Times again with a piece on "The first computer programmer was a woman - Ada Lovelace"
  • Salon with an article: "The State College of Florida recently scrapped tenure for incoming faculty. New professors at this public university will be hired on the basis of annual contracts that the school can decline to renew at any time."
  • Reason on generation obnoxious: ""We have spoken. We are speaking. Pay attention." Nothing captures the attitude of the modern college activist as perfectly as this statement, made by Yale University students petitioning the English department for changes to the curriculum (they wanted to read fewer white male poets).It's the constant refrain of the far-left social justice student: Our minds are made up. The time for discussion is over. We aren't here to be educated. We are here to educate you."
  • Huffington Post on the US winning the math olympiads for the 2nd year in a row.

SageTex: Polynomial Interpolation 1

FibSeq

Vacation! But before it's time to go I've added a problem to the SageTex: Matrices page. It's not enough to have a lesson on polynomial interpolation like was mentioned earlier so I've put together a problem. Given what starts out as the Fibonacci sequence, students will have to find a polynomial to justify the sequence continuing with a term which breaks the pattern that people think is there.

Here are some stories that caught my eye this past week:

  • The Daily Caller has noted "The University of North Carolina at Chapel Hill issued a guide this week which instructs students that Christmas vacations and telling a woman “I love your shoes!” are “microagressions.” The taxpayer-funded guide — entitled “Career corner: Understanding microaggressions” — also identifies golf outings and the words “boyfriend” and “girlfriend” as microagressions...Christmas vacations are a microagression, the public university pontificates, because “academic calendars and encouraged vacations” which “are organized around major religious observances” centralize “the Christian faith” and diminish “non-Christian spiritual rituals and observances.”"
  • NJ1015 with a tragic story, "The 17-year old Robbinsville High School student who fatally ran over this district’s beloved schools superintendent in April was talking on her cell phone at the time of the crash, prosecutors said Thursday.The student, who is now 18 but is not being named by authorities because she was a juvenile at the time of the crash, has been charged with second-degree death by auto and leaving the scene of a motor vehicle accident resulting in death.She also was ticketed for reckless driving, improper use of a cell phone while driving and leaving the scene of an accident, the Mercer County Prosecutor’s Office said."
  • ZeroHedge with a piece on Randi Weingarten and teacher unions, "Weingarten instructed investment advisers at the federation's Washington headquarters to sift through financial reports and examine the personal charitable donations of hedge fund managers, focusing on those who want to end defined benefit pensions, and entities backing charter schools and the overhauling of public schools. In early 2013, the union federation published a list of roughly three dozen Wall Street asset managers it says donated to organizations that support causes opposed by the union, and the federation wanted union pension funds to use the list as a reference guide when deciding where to invest (or not invest) their money. Said otherwise, if asset managers don't support unions, the unions won't invest with the funds." You'll have to go to the link to see what happened next.
  • Philly.com with a story that's gotten a lot of coverage, "On June 16, police were called to an unlikely scene: an end-of-the-year class party at the William P. Tatem Elementary School in Collingswood. A third grader had made a comment about the brownies being served to the class. After another student exclaimed that the remark was "racist," the school called the Collingswood Police Department, according to the mother of the boy who made the comment....The boy's father was contacted by Collingswood police later in the day. Police said the incident had been referred to the New Jersey Division of Child Protection and Permanency. The student stayed home for his last day of third grade."
  • Campus Reform has a story on "Donald Trump’s campaign slogan is a “racialized, targeted attack” according to Skidmore College’s Bias Response Group (BRG). Three white board messages with the phrase “Make America Great Again” were included in the BRG’s annual report and classified as “written slur[s] or graffiti” because they had been written on the white boards of female faculty of color." These Bias Repsonse Groups provide flimsy cover to assault free speech.
  • WGBH news reports on the Harvey Silvergate's Muzzle Awards for 2016. First place went to Yale over "...a string of events that started with an email about Halloween costumes, Nicholas and Erika Christakis, two well-respected professors at Yale University, resigned their administrative posts as faculty of Silliman College (a dormitory) amid student protests."

Brexit: because statistics isn't really math

A last minute change in my post due to Friday's historic day that wiped out 2 trillion off of the worlds equity markets. The cause? Virtually all the experts predictions were wrong on Brexit: the pollsters got it wrong, the betting line was wrong, and the markets got it wrong-- thank the statisticians for another job well done! CNN reports, "Ahead of the 2012 U.S. elections, Nate Silver, from the website FiveThirtyEight, correctly predicted who would win all 50 states, even as pundits were saying the race was "too close to call." In 2008, he had also correctly projected all but one state. As this year's British election results started trickling in, Silver tweeted that the world "may have a polling problem." "Polls were bad in U.S. midterms, Scottish referendum, Israeli election and now tonight in UK," Silver said....In a commentary on FiveThirtyEight, Silver suggested that forecasters had been overconfident. "Polls, in the UK and in other places around the world, appear to be getting worse as it becomes more challenging to contact a representative sample of voters. That means forecasters need to be accounting for a greater margin of error," he said....Prediction models for the U.S. elections had also become more reliable, Parakilas said, something he didn't believe had happened yet in the UK...." And while they tell you NOW that the models in the UK are less accurate, the Nate Silver that CNN trumpeted as the expert is the same Nate Silver I mentioned here, who performed so badly.

The great thing about statistics is you can explain why you were wrong after you learn that you were wrong. Here are some other explanations floating around as to why the results were so wrong:

  • there was a lot of rain in London and that could impact the turnout of city dwellers who tended to support Bremain.
  • Brexit voters were criticized as racists and not too bright, as all the "experts" came out for Bremain. As a result, Brexit voters were not honest when polled. I think this is a big factor, and a reason why applied statistics is more of an art than science.
  • the inability to predict how many people would turn out to vote
  • Bremain is what the establishment wanted to win so the dissenting voices were minimized. This seems unlikely given how genuinely surprised the establishment was.
  • From Reuters, "Predicting the outcome of Thursday's referendum was harder than that of a national election because there was virtually no historical data to draw on, said David Rothschild, an economist at Microsoft Research. He said pollsters also did not pay enough attention to working class and less educated voters....Rothschild, who also is a fellow at Columbia University’s Applied Statistics Center, said he expected forecasting to improve with a transition from polls using small, random representative samples to large Internet-based ones with rich demographic data. "If I have one million respondents with a large amount demographic data, I should be able to predict outcomes better, or I'm not a very good statistician," he said." OK, but right up to the end statisticians were confident of the result even though many polls were based on only a few thousand people. How representative were the samples they claimed were representative and why were so many experts wrong.
  • Statsblog.com gives their 5 reasons, including "Survey respondents not being a representative sample of potential voters (for whatever reason, Remain voters being more reachable or more likely to respond to the poll, compared to Leave voters)". Hard to argue with the samples not being representative of the whole AFTER you've been proven wrong. But aren't statisticians supposed to be guarding against this happening?

Lots of excuses; of course some or all arguments could be right BUT:

  • From NYTimes, "Britain’s decision to leave the European Union on Thursday was a big surprise. As late as 6 p.m. Eastern in the United States, less than five hours before the results became clear betting markets gave “Remain” an 88 percent chance to win the election, but it wound up losing by four percentage points....One could certainly argue that the polls were “wrong” in the sense that they tended to show a slight Remain advantage heading into the vote count. But it was clearly a distinct possibility that Brexit would win, based on the available survey data. So it’s hard to argue that this was a big polling failure, and it’s a bit strange that the financial markets appear to have been caught completely by surprise." The American Thinker responds to this stupidity,  "Sure it wasn't.  Once you are done rolling around on the floor in laughter at claims that this wasn't a massive polling failure, read on....Not a single one of the well known polling aggregators/predictors picked Brexit in their last-minute final projections...Thus, we had a systematic bias in the aggregated polling data that ranged from 4% to almost 11%.Individual polls leading up to the vote were publishing ridiculous results.  In the week prior to the vote, 9 of the 13 polls predicted a victory for Remain ranging from 1% up to 10%.  Just three polls had Leave in the lead, but just by 1% to 3%– i.e., still below the actual margin of victory – and one poll had a tie.  Not a single individual poll got the result correct, or overpredicted a Leave win....The overall bias in favor of Remain was effectively uniform, which is statistically impossible if the bias was random.  The bias was systematic."
  • In region after region the Brexit numbers were consistently underestimated by several percentage points.

The Telegraph has a lot more analysis with charts and graphs as to what wrong, "Professor Curtice was cautious throughout the campaign, saying that“some of the polls are definitely wrong” in “a cloud of uncertainty”. There were also clear distinctions between phone and online polls - phone polls invariably scored higher results for Remain compared to online." [Comment: Aren't you glad you know this now, after the fact?] "Interestingly, the Leave vote remained constant across both phone and online - it was the “don’t know” score that decreased for phone polls, and Remain seemed to be securing most of these. This was falsely encouraging for the final result. According to YouGov’s analysis, the reduction of don't knows for phone polls was because people were more likely to give an opinion when in conversation with someone, rather than admitting they didn’t know what they thought about such an important choice.Analysts dismissed the idea that different methods would reach different demographics.." So they got it wrong again. But at least their hindsight is 20/20.

Some lessons according to the Washington Post,

  • "First, we did not see this coming. For some weeks now, Stephen Fisher and Rosalind Shorrocks have been tracking referendum forecasts. They consider a wide range of sources, from forecasting models based on polls, to citizen forecasts, to betting markets. None of these methods saw a Leave outcome as the most likely outcome."
  • "Second, this was not a systematic polling failure of the same magnitude as last year’s U.K. general election, where opinion polls badly underestimated the Conservatives’ chance of victory." So the defense is: the failure isn't as bad as when they REALLY messed up last year. That should inspire confidence.
  • "Third, we learned something about campaign dynamics in referendums — and we went wrong by believing too firmly in a claim about how voters decide. Part of the disparity between relatively close polls and relatively confident betting markets was due to the belief in status quo reversion — the idea that undecided voters will be more likely to choose the status quo option (in this case, Remain) than the alternative."
  • "Fourth, given the types of areas that voted to Leave, and given the available polling evidence, it seems likely that a majority of Britons have traded economic benefits for restrictions on people from the European Union coming to live and work in Briton. The areas which voted Leave were older, whiter, and less likely to have a university education."

Another black eye for statistics and statisticians but you can't expect statistics to have the accuracy of mathematics--it isn't math any more than mathematical economics is.

Here are some stories that caught my eye last week:

  • ZeroHedge reports "The percentage of new doctorate recipients without jobs or plans for future study climbed to 39% in 2014, up from 31% in 2009according to a National Science Foundation survey. Those graduating with doctorates in the US climbed 28% in the decade ending in 2014 to an all-time high of 54,070, but the labor market - surprise surprise - has not been able to accommodate that growth. "The supply of PhD's has increased enormously and the demand in the labor market has increased but not nearly as fast. When you can import an international workforce or outsource research, you have a buyer's market" said Michael Teitelbaum, senior adviser to the Alfred P. Sloan Foundation."
  • Wow! RT notes "Around one in 10 of the students attending the largest four-year public university system in the US is homeless, while one in five cope with food insecurity, according to a new study by the California State University system."
  • I've got nothing against unions, my issue is against the terrible decisions/policies they (or anyone) support. Case in point. ChicageCBSlocal reports "He crossed the line – the CPS teachers’ one-day strike — out of his love for the classroom. Joseph Ocol stuck with his kids and brought home a chess championship. Tonight, he’s expelled from the union and wonders if he’ll even have a jobCBS 2’s Brad Edwards reports.The union’s decision came via certified mail, in a letter signed by Chicago Teachers Union President Karen Lewis.....the CTU said in a statement: “Mr. Ocol has been informed of his member privileges and is talking to us through media, which is unfortunate. All members are well aware of what happens to strike breakers and are informed by their own peers of the process for both suspension and reinstatement. CTU is a democratically led member-organization.”"
  • CounterPunch explains how Common Core helps bust the unions.
  • HeatStreet looks into how colleges are letting students censor speach, "For many students and professors, one of the great appeals of college life is being exposed to new and different ways of thinking. But that age-old process is now under threat at schools around the country. Take the University of Northern Colorado. After two of the school’s professors asked their students to discuss controversial topics and consider opposing viewpoints, they received visits from the school’s Bias Response Team to discuss their teaching style. The professors’ students had reported them, claiming the curriculum constituted bias. These incidents, both in the 2015-2016 academic year, reflect a growing trend in higher education. College students increasingly demand to be shielded from “offensive,” “triggering” or “harmful” language and topics, relying on Bias Response Teams to intervene on their behalf. Such teams are popping up at a growing number of universities....To date, more than 100 U.S. public colleges and universities have established Bias Response Teams."
  • HeatStreet again with "Kayla-Simone McKelvey will serve 90 days in jail, five years of probation and 100 hours of community service for her role in a racially charged hoax threat issued to Kean University students. McKelvey, who is black and the former president of the New Jersey college’s Pan-African Student Union, used a fake Twitter account to send a message threateningto kill a group of black students at an on-campus rally in NovemberThe Twitter account, @Keanuagainstblk, claimed that the anonymous user would “kill all male and female black students” at Kean and issued a bomb threat against the school. The account was quickly suspended from Twitter, but not before causing an uproar on social media. Supporters of #BlackLivesMatter across the country called on the university to take action to protect protesting students, and demanded that Kean President Dawood Farahi resign. They tried to use the threat to demonstrate that Farahi had not done enough to diffuse racial tension on campus....McKelvey told the court she was sorry she issued the threat, and that she still believes her actions helped expose racism on campus...But if McKelvey’s excuse sounds a bit strange, she’s not alone, even at Kean, in thinking that her clearly illegal actions “helped” fellow social justice warriors to bring Kean’s “systemic racism” to light. Some Kean students said that the threat’s author didn’t matter that the threat was still evidence of strong racial bias on campus."
  • LA Times reports that Pat Haden used an educational foundation to enrich himself and his family, "Under Haden's leadership as board chairman, however, the $25-million foundation became a lucrative source of income for him and two of his family members -- even as its scholarship spending plunged to a three-decade low and the size of its endowment stagnated, a Times investigation has found.Haden, his daughter and sister-in-law together collected about $2.4 million from the foundation for part-time roles involving as little as one hour of work per week, according to the foundation’s federal tax returns for 1999 to 2014, the most recent year available. Half of that, about $1.2 million, went to Haden. His annual board fees have been as high as $84,000; the foundation paid him $72,725 in 2014....Many foundations do not pay their board members, philanthropy experts say. The $1.5-billion California Community Foundation, for example, does not pay board members. Foundations that do compensate board members, those experts say, typically pay far less than the amounts received by Haden and his relatives. The $12.5-billion Ford Foundation paid its board chairman about $30,000 less than the Mayr foundation paid Haden in 2014. Mark Hager, an associate professor of philanthropic studies at Arizona State University, said in an email the Mayr payments to the board would be high “even for a foundation that was giving out more than $50 million in grants each year.”“I’ve never heard of fees that large,” said Adam Hirsch, a law professor at the University of San Diego who specializes in trusts."