Thursday, October 27, 2016

Another failure of scientific peer-review: a wrong paper on the energy return of photovoltaic energy

Theoretically, whatever is published in a scientific journal should go through a rigorous review process that ensures that it is correct and reliable. Unfortunately, it doesn't work that way.

If you follow the debate on renewable energy, you know how important is the question of the energy return (or EROEI) of the various sources. An EROEI lower than one would make the source - PV, wind, or whatever, an energy sink, not a source. And this is exactly what Ferruccio Ferroni and Richard Hopkirk have been claimed with a paper recently published in "Energy Policy" that arrives to results that are completely different than to those of all the other studies on the subject.

The paper by Ferroni and Hopkirk is simply wrong. You can read below a complete demolition of their arguments performed by Maury Markowitz. But, no matter how wrong is the paper - and it is wrong - this story raises some disturbing points about how scientific information is validated and diffused.

1. Any paper, no matter how bad, poorly conceived, and ultimately totally wrong, can be published in a scientific journal if the authors are persistent enough and try many times. Eventually, they will find a combination of editors and reviewers sufficiently incompetent, sloppy, or biased that they will accept it.

2. There is no way to correct the mistakes of a wrong paper once it is published. The journal will retract it only if it is possible to prove that the authors are guilty of evident fraud or plagiarism. But "simple" mistakes, things such as wrong citations, misinterpreted data, inappropriate data treatment and the like are rarely sufficient to force retraction.

3. The only way to protest against a wrong paper is to ask to the journal to publish a rebuttal. They will do that with the same degree of willingness that you feel about having a tooth pulled, but they will do that, asking also to the authors of the original paper to write a counter-rebuttal. The whole task is long, painful, and ultimately useless as it may end up giving more visibility to the initial paper.

4. Mark Twain is reported to have said that "A Lie Can Travel Halfway Around the World While the Truth Is Putting On Its Shoes". That's exactly what happens when a wrong paper sees the light in a scientific journal. It will spread fast with the people who are seeking for whatever can help them with their confirmation bias. And the rebuttals will be considered as proof of the conspiracy by the PTBs to suppress the Truth.

That's exactly what's happening with the F&H paper, gleefully paraded around as proof that photovoltaic energy is a scam and a waste of money. A rebuttal to the paper is in preparation by a group of scientists, but it will arrive late and will do little to correct the wrong information already diffused on the Web. The problem is that this information affects choices that will determine our future: we can't afford to base them on wrong studies that somehow managed to get published.

So, how did we find ourselves into this mess? Who created a scientific review system that has no quality standards, no independent quality control, no audits, no nothing? I have no idea, but it is clear that the system badly needs a serious reform.

Here is the demolition of the Ferroni and Hopkirk paper, reproduced from Markowits's blog


From Energy Matters by Maury Markowitz

Another PV ERoEI debacle May 17, 2016

Posted by Maury Markowitz in balonium, solar.
Tags: , trackback


Your face should have this expression when you read Ferroni and Hopkirk’s paper.

recent report by Ferroni and Hopkirk explores the energy balance of solar power, and concludes that using PV is energy negative. That is, building PV requires more energy than the panel will produce over its lifetime.
Claims like these pop up from time to time, and normally end up being based on definitional tricks on the part of the authors. This example is no different in that respect, but in this case they also add a liberal dose of bad data.
The paper is so filled with errors and omissions that’s it’s almost breathtaking. Once again, dear reader, it’s time for the deep dive.

The sincerest flattery

While googling myself (I can’t remember my URL any more than you can) I found to my delight that the name of this blog has been taken up by a pair of bloggers from Aberdeen. How I never came across this previously is something of a mystery; I guess the web is deeper than one would imagine.
In any event, a May 9 post by one of the authors pointed me to the paper that is the topic of the rest of this article. After stating that the topic of ERoEI is new to the blog, he goes on to note that when he came across this paper, “the findings are so stunning that I felt compelled to write this post immediately.”
When I come across a study in the renewables field with findings that are “stunning”, I normally hold it at arm’s length until I can run the numbers myself. That’s because the field is utterly filled with bogus information from thinly disguised coal company shills to the nuclear true believers

Don’t get me wrong, there’s just as much BS going the other way, from the usual suspects to the space heads, which is all the more reason to be super-skeptical. While Mr. Mearns does make some comments about the validity of certain inputs in theoretical terms, in the end, he quotes the bottom line:

Solar panels will produce only 0.83 times the amount of energy they take to produce… If correct, that means more energy is used to make the PV panels than will ever be recovered from them during their 25 year lifetime.
That’s a big “if correct.”
And guess what, it’s not correct.

Start bad…

So let’s get into the meat of it. The paper starts with the authors having examined 28 other papers on the topic and found they had a wide variability of Cumulative Energy Demand (CED), the amount of energy used by a product over its lifetime. They conclude that “the authors … were not following the same criteria in determining the boundaries of the PV system.”

Now getting the CED is important, because the overall energy balance, ERoEI,i s basically energy out divided by energy in. So you’re going to need to have a good value for that CED, and there’re all over the map. So their solution is to define an entirely new version – yay!
But now they change gears, and work on the other side of the equation, the total energy produced. 

And they attempt to do this in per-square-meter terms.
Now stop right there.
The industry, and I mean the entire power industry here, not just the renewables industry, measures everything in either per-watt or per-kilowatt-hour terms. That’s because the physical mechanisms of the generators differ wildly, but a watt is a watt, so when you convert to those terms you have a real apples-to-apples comparison.
Consider an example; if I tell you a hydro dam cost $2/Watt and a new wind turbine costs $1.50/W,well, there you have it. Now what if I told you that the dam cost $2 per square meter and the turbine $10? Well, does that area include the reservoir? Does the turbine include all the area around it, or just the actual footprint on the ground? See the problem? Area is tough to pin down. Dollars are not.

So why would the authors pick such an odd unit? I can’t say for sure, but in the abstract they mention something about how “solar radiation exhibits a rather low power density”. Well, sure, and that’s important why? Apparently it’s not, because it only figures in very peripherally in the calculations, and has no effect on the bottom line.
Whatever, let’s get to the numbers at hand:

The data are available in the Swiss annual energy statistics … and show an average value of 400 kW ht/m2 yr (suffix “t” means “thermal”) for the last 10 years. This is an indication of the rather low effective level of the insolation in Switzerland. … The uptake from the incoming solar radiation is converted into electrical energy by the photovoltaic effect. The conversion process is subject to the Shockley-Queisser Limit, which indicates for the silicon technology a maximum theoretical energy conversion ef- ficiency of 31%. Since the maximum measured efficiency under standard test conditions (vertical irradiation and temperature below 25 °C) is lower, at approximately 20%, the yearly energy return derived by this first method in the form of electricity gen- erated, amounts to only 80 kW he/m2 yr.
Ok, if you don’t really know much about solar power, you might not immediately see the problem with this statement. What they’re doing is a double conversion – they’re not calculating the amount of energy produced by a PV panel, they’re taking the amount of heat collected by a thermal panel, then applying a formula to convert that to expected electrical power production.


But that conversion is totally wrong. The Shockley-Queisser Limit doesn’t work on thermal energy, it works on the original solar energy. And no, the thermal energy is not a good proxy for the original solar energy. The main contribution to the losses in PV, the SQ limit, is wavelength, which doesn’t come into play in thermal collectors at all. And the main contributor to losses in thermal collectors is ambient temperature, which has a minor effect on PV.
The two are just not the same, you can’t do that.
But more to the point, why would they do that? Because the same source they quote for the thermal value publishes actual electrical output figures as well, which they then go on to quote:

According to the official Swiss energy statistics (Swiss Federal Office of Energy, 2015), an average for the last 10 years of 106 kWhe/m2 yr is obtained for relatively new modules.
This number is 30% higher than their calculation based on thermal, a discrepancy they don’t even try to explain.
Beyond that, any number that is “an average for the last 10 years” is, by definition, not talking about “relatively new modules”. Ten years ago the average panel was about 160 Watts and cost about $5.00/Wp. Today they’re around 280 Watts and cost about $0.45/Wp. The vast majority of the world’s PV was installed in the last three years, so any calculation based on data older than that is just plain wrong.


And even this number, 106, is significantly lower than 

I would expect given that Switzerland has fairly average insolation for mid-latitude Europe. So what’s up with that?
Well when I checked the cited source I found that no such number is actually reported. One can only find total output numbers and then work back from there, but the authors fail to give their calculation. And those totals  -wait for it- go back over a decade, so we’re right back to that problem again.
Which is all the more funny when you consider that such data is trivially available on the ‘net. Anyone who wants do to do this calculation should do what we all do; use NREL’s PVWatts. It has highly accurate weather data going back 30 years taken only from first-class sensors.
I typed in Zurich for the location, and selected the TMY3 data set. For the system size, I considered a typical modern 280 Watt panel at 1.6 m², or 175 W/m², and typed 0.175 into the System Size. I also changed the tilt from 20, which is good for California, to 30, which is good for Zurich. And here it is:

Screen Shot 2016-05-16 at 10.05.02 AM.png
They said 106. We’re at the very first number in the paper, and they’re already off by a factor of 60% from what the industry standard tool suggests.No attempt is made to explain this, except for dismissive comments about industry calculators.

 …get worse…

Now the authors turn their attention to expected lifetime of the panels, which is needed in order to calculate the overall lifetime energy production. They do so in a rather convoluted fashion, starting by considering the amount of panels recycled in Germany:

This was 7637 t. A module of 1 m2 weighs 16 kg and 1 kWp peak rating needs 9 m2 and consequently, scaling this up, a 1 MWp module will weigh approximately 144 t.
Hmmm. A SolarWorld 280, a typical modern panel, masses 17.9 kg. That’s 17.9/1.6 = 11.2 kg/m². I really have no idea where they got their value, and they don’t include any sort of reference. A 1 MWp system using these panels would require 1 million / 280 ~= 3570 panels, or 3570 x 17.9 = 63,903 kg = 64 t. So now we’re at calculation number two, and we find they’re off by another factor of two.
The paper goes on to use these numbers to suggest a real lifetime is about 17 years. Now the problem is that if older panels are heavier, then the number on a per-kg basis is automatically skewed towards older panels again. Or to put it another way, if you had 10 panels from each year since 1990 and scrapped one from each year, when measured by weight it would seem that more older panels are dying.
And once again I’m left scratching my head why they would use this convoluted magic, when one can find real values in seconds. In fact, one of the most quoted examples is right up the road from them on the LEEE-TISO buildings. The vast majority of these panels, apparently the first grid-tie system in the world, are still running fine after almost 35 years now. They calculate the losses at 0.2 to 0.5% a year, which corresponds to a panel lifetime on the order of 60 years.

…a little more…

They then ignore their previous calculations, and use a 25 year lifetime. So apparently all of that was for nothing! And that brings us to this:

Experience has shown that, on average, efficiency and hence performance de- gradations of around 1% per year of operation must be expected (Jordan and Kurtz, 2012).
Now we go from bad to terrible. They claim this 1% number comes from a paper by Jordan and Kurtz. Well that paper is available online, and actually states the measured rate varies widely, from 0.23% to as much as 2%. And the mode among that data is between 0.4 and 0.5%, which you can see on page 4 of the paper.
So if the paper they quote says it is 0.5%, how do they get 1% from the same report? Because they chose the figure on the right of page 4, which includes low-quality data. And what is the difference between the two? Well, the low-quality data is:

very sensitive to several sources of error that could skew the results. Soiling, maintaining calibration and cleanliness of irradiance sensors, module baseline data (nameplate vs. flash test), and not appropriately accounting for LID are just a few major sources of data errors.
In other words, the high-quality data is based on controlled measurements, where they account for these effects and report only the actual panel degradation. In contrast, the low-quality data does not account for these issues, so it includes all sorts of external environmental effects. They fail to mention any of this, they knowingly use the bad data.

They also fail to mention that while the 1% value was indeed used by the industry in the past, they number the industry now uses is 0.5%. And that’s because a number of long-period studies demonstrated 1% was too high. In particular, a NREL study found that panels made before 2000 had a degradation rate of 0.5%, and those after 2000 fell to 0.4%. That indicates the sorts of improvement processes that continue to this day. And, of course, they have the LEEE-TISO numbers, which strongly agree with both of the sources quoted above.
Ferroni and Hopkirk then claim:
There are also other, external factors, which can reduce PV module lifetime, for instance the site, the weather and indeed climatic conditions. These aspects do not appear to have been treated in the scientific literature in connection with photovoltaic energy usage.
Oh come on! They actually talk about these factors in the paper they’re quoting! These sorts of effects are also considered in every tool that predicts output, including PVWatts. And what, do they think their weather would be any different than the LEEE-TISO install down the road from them? Ugh.

…which brings us to…

Ok so now all of this feeds into this equation:
Screen Shot 2016-05-16 at 10.46.51 AM
What this does is add up all the yearly power production figures over the lifetime of the panel to produce the total energy output of the panel. And using their figures they get 2203 kWhe/m².

Ok, just for funzies, let’s run the exact same equation,but we’ll use NREL’s 30-year climatic data, and the industry-standard 0.5% degradation. That gets you 3795 kWhe/m². Almost double.
And I need to point out that I’m using industry standard numbers, and in one case, from the same paper they quote. Their result is lower simply because they have selected worst-case-scenarios for all of these numbers. Normally one would indicate this with error bars or using the mode or mean values, like I’m doing here, but they haven’t done that. They just say these numbers are correct. They aren’t.

So, now, the other side of the equation

Ok, so the authors have now developed a number for the total output of the panel, now it’s time to consider the total energy input. And that starts like this:
The average weight of a photovoltaic module is 16 kg/m2
As I noted earlier, the SolarWorld example I linked to above is 17.9 kg for 1.6 m², or 11 kg/m². I assure you this is typical, but feel free to Google “solar panel weight” if you don’t believe me. And then they go on to state:
and the weight of the support system, inverter and the balance of the system is at least 25 kg/m2
25 kg for every square meter? I’ve installed a number of crazy systems, and I can assure you, we never came even close to that. Invariably the heaviest part was the panel.

So let’s check on their sources. Well, first of all they don’t actually quote the original source for those numbers, they quote a University of Toronto thesis from 2009 where you’ll find that:
Support structures for PV panels are made from aluminum or steel, with the majority of systems using steel.
The majority use steel? Uhh, no. And the 25 kg/m² figure in there? It comes from two even earlier papers from 2007. And when I looked there, the one that did have the 25 figure was quoting that from the other, which didn’t have that number in it. I really have no idea where it comes from.

There’s only so much time we can spend on that madness. So let’s just use the power of the internet to find modern values. Check out page 6 of this fairly modern product guide to mountings, which puts the total weight of mounts and panels at 16 kg/m². If we use the modern figure of 11 kg/m² for the panel, that puts the weight of the support structure at 5 kg/m². That same guide also includes values up to 50 kg/m², but that’s for ballast on flat roofs, which are concrete blocks, not steel. This is not used on sloped roofs or ground mounts, but as it might represent as much as 15% of the market, you can factor that in as you wish.

Ok, let’s keep going.
16 kg (module) + 25 kg (balance of plant) + 3.5 kg (significant chemicals) = 44.5 kg/m2
Ok, let’s use our numbers from real sources instead: 11 + 5 + 3.5 = 19.5 kg/m²
Which brings us to:
Since the total lifetime energy return is 2203 kW he/m2, we obtain a material flow of 20.2 g per kWhe
Maybe. Or maybe it’s 19,500 / 3795 = 5.1 g/kWh? Once again, using numbers from the industry I get a number four times “better” than they do.

Now why is this important? Because that number is basically how you calculate the energy needed to make the panel and rest of the system. So much weight of steel takes so much energy to make, and so forth. So if you reduce that by four, you’re almost reducing the CED by four, right off the bat.

Show me da money!

So now the authors move onto the “use of capital.” The basic idea here is that money embodies energy, in a way. Basically everything requires energy to build and ship to you, so if you spend $1 on something, some of that is paying for that energy. So, on average, you can say that a dollar of capital has a certain average energy content, which for convenience, we’ll express in kWh.

So if we’re going to start down that road, we need to have some sort of value for how much capital we need. Here’s the relevant part:
The actual capital cost for a sample group of fully installed PV units, 2/3 roof-mounted and 1/3 free-field-mounted, in Switzerland lies at or above 1000 CHF/m2 with large cost variations of up to 30%, due principally to the uncertainty in the price develop- ments of PV modules. The NREL (National Renewable Energy Laboratory of the U.S. DOE) reports capital cost for fully installed PV units in the lower end of the price range given above. The 1000 CHF/m2 cost, translated into specific cost for installed peak power is 6000 CHF/kWp and is a result of personal experience of the authors.
Ok ok, let’s take this bit by bit. First they have a 2/3 roof and 1/3 field split. They don’t provide a source,of course. I’ve never seen numbers anything like this, and it is trivially easy to find industry values that show the opposite.

For instance, even in Germany where the majority of installs were residential, they represent only 35% of the total buildout. In the US, where the split used to be about 50/50, utility installations now far outnumber residential. Now I mention this because utility installs are ground-mount, so according to these recent sources, the total installed base should be at least the opposite of what they use in this paper.

And this is important, because the capital cost of the system is roughly double for roof mounts, especially residential. That’s because you’re installing far less panels per job, so setup and administration is a lot more on a percentage basis. And for that reason, utility scale installs are dwarfing residential these days, a move that continues to accelerate every year.

Which brings us to the second number, the actual capex value they will use from here on in. That number is 1000 CHF/m2, but that translates into 6000 CHF/kWp based on the “personal experience of the authors”?!

Really. In a peer reviewed paper, we’re being told just to take their word for it. Wow.

Well they can’t be bothered to cite their numbers, but I’ll cite mine. I will refer to the most comprehensive and up-to-date industry-measured values one can easily get, the yearly Lazard LCoE report. And that number, averaged across the western world, is found on page 11, and it is $1500/kWp for utility and $3500/kWp for residential.

Using the modern 1/3rd residential, 2/3rds commercial split, that gives us (3.5*.33)+(1.5*.66) = $2145/kWp average. Now to make a kWp of panel using those SolarWorlds, we need 1000 / 280 = 3.57 panels, and since each panel is 1.6 m², we need 3.57 x 1.6 = 5.7 m², so on a per-m² basis that’s $2145 / 5.7m² = $376 / m².

That’s less than 1/3rd the number they’re quoting, although they do so out of thin air. Even this number overestimates the contribution of residential installs moving forward. I prefer to use the utility rate as more indicative of the real capex of PV; $1500 / 5.7m² = $263 / m².

Working in a coal mine

The paper then moves onto breaking out the various components of that cost and calculating the energy value of each one. They start with labour. After quoting four year old figures, they say:
Based upon the authors’ experiences for typical local labour costs per square meter of PV module are: project management (10% of capital cost), installation (506 CHF per m2), operation for 25 years, including insurance (1.67% of capital cost per year for 25 years) and decommissioning (30% of installation). The total labour costs amount to 1175 CHF/m2.
Now in case it’s not obvious, I want to point out that all of these measurements are based on the capital cost. So if your capital cost is off, this is too. And their capex is off by a factor of three to four. Because, once again, it’s just “the authors’ experiences”.

And to make my point, consider that value in the middle, the installation costs. They’ve already said that the total capex for the system is 1100 CHF / m², and here they say that installation labour is over 500 of that, roughly half.

Really? According to these numbers, published only months ago, all soft costs put together cost around 52% of the system price. And you’ll note that number puts all-in prices at 1300 Euro, basically identical to the Lazard number at $1500 US, and, once again, 1/4 the number Ferroni and Hopkirk create out of thin air.

So for the moment, lets ignore their made-up numbers and use these industry standard ones. They calculate 505 kWhe/m² based on 1175 CHF/m² of labour. Using these figures we see that all the soft costs are 52% of 1300 Euro per kWp, or 748 CHF/kWp, or 209 kWhe/m².

But what’s another factor of two between frenemies?

And finally, in section 5.5.3, the duo calculate the energy value of the capital itself. Basically the idea here is that if you have to borrow the money (or you can flip that to opportunity cost, same thing) then you could express that in terms of panels you could have bought with that interest (so to speak). You can think of that as “lost energy” in a way…
  • Using their rate of 1100 CHF/m², they get a value of 420.
  • Using the industry rate above, 1300 Euro/kWp, that gets us around 120.

Show me da money (again)!

Which brings us, finally, to their totals in Table 4:
Screen Shot 2016-05-16 at 12.13.34 PM
Now let’s do the exact same thing using the numbers we’ve calculated:
CED 1300
Integration 349
Labour 209
Faulty equipment 90
Capital 120
Total 2068

So basically, just considering the known-good, widely-available capex number, we’ve reduced the “energy investment” by 22%.

All of this goes back to the original claim. They claim that the ERoEI is 2203/2664 = 83%. But a whole lot of that is made up by the cost of capital, based on a bogus number. By changing that one number to the one actually measured in the field, we get 2203/2068 = ERoEI 1.06.

And if we instead insert NREL’s number for the insolation, and use the industry standard degradation, it becomes 3795/2203 = ERoEI 1.7. That’s better than fracked oil in the US.

And we haven’t even touched that CED number, which, as you can see above, is based on some rather odd numbers about system weights.

That’s not all folks…

Now we come to the issue of recycling. In the calculations in the paper, the authors consider the panels to have a 30% decommissioning fee, which is added in the labour term.

But they totally ignore the salvage value of the panels. Panels are basically glass, aluminum, some silver and some copper. People pay for these things, which is precisely why the Europeans have a recycling program for panels.

Given an average 50% energy recovery for recycling, we can reduce the CED of a 2nd generation panel to 650. Running the same calculation gets us 1419, so 2203/1419 = 1.55, or 3795/1419 = 2.7.

And if you do consider the recycling as a potential revenue stream, then the labour line is reduced by some portion of that 30%, which brings the denominator to 1340. And that gives 2203/1340 = 1.64 or 3795/1340 = 2.83.

So in the end

Consider this: the calculation they use in their paper would produce different results if the interest rate changes, the FX rate between the Yuan and Swiss Franc changes, or the price of installations continues its astonishing downward fall.

So, what exactly is this figure measuring?

It’s certainly not measuring anything like the “embedded energy content” of the panels. That wouldn’t change just because someone types a number into a Bloomberg terminal. Yet that’s precisely what happens using their calculation.
And finally, I need to point out the glaring fact that the authors don’t run the same calculation on any other power source. Given that sources like nuclear are far more capital intensive than PV (which is why no one is building them) their calculation of “ERoEI” is worse.

 This paper is just plain bogus. The entire methodology is based on numbers that have no physical reality (money) and the authors deliberately cherry pick data to make those numbers “prove” their point, or just make up values out of the air. All of this is glaringly obvious, and is simply yet another example of the sorts of attacks renewables face at the hands of the true believers in the nuclear field.


Ugo Bardi is a member of the Club of Rome, faculty member of the University of Florence, and the author of "Extracted" (Chelsea Green 2014), "The Seneca Effect" (Springer 2017), and Before the Collapse (Springer 2019)