I got a reduction of approx 10 dB at the high frequencies. The roll off started at approx 500 Hz. ... Interestingly the dB levels at approx 120Hz were 3-4 dB higher than without the gobo,
Yup. That sounds about right... It really does get louder in the low end, for complex acoustic reasons, and even in the high end you don't get usable reduction. That's typical of that type of solution, and frankly, there's nothing you can do about it.
A typical singer can sing at around 80 to 90 dBC, so your 10 dB drop in the high end would only bring that down to maybe 70 dB in the drum mics: still very loud. Plus you have the low frequency amplification issue, so the total isolation is lower.
A few years ago, Sound on Sound magazine ran a series of tests on several commercial products that attempt to do the same as you tried, for product with names such as "Reflection Filter" and " Portable Vocal Booth", and they came to the same conclusion you did: this solution does not work.
Here's a link to the article:
https://www.soundonsound.com/reviews/ho ... cal-booths The test was done in part by one of the world's leading acousticians, very well respected author of many books and papers, Prof. Trevor Cox. You'll notice that you got the same results they did when they tested in a proper acoustical test laboratory. Take a close look at the response graphs in Figure 1... And the explanations are quite clear. Well worth reading that article.
Any ideas on whether some proper high density Rockwool or fibreglass would get me in the ballpark, or is this something that I just need to buy a bag and try it out?
It is entirely predictable by acoustic theory, so save yourself the time and money, and DON'T buy that mineral wool!

Contrary to popular belief, insulation does not stop sound. It damps resonance, and absorbs sound, but that's not the same as stopping sound. That's not intuitive, but it is reality. A lot of things in acoustics are not intuitive. You cannot use insulation to block sound, because the laws of physics don't allow it.
If I could get a 10-20 dB reduction from 150Hz upwards, this could be worthwhile.
In order to stop sound, you need mass: heavy, thick, solid materials, such as wood, glass, brick, concrete, steel plate, etc. There's a simple equation in physics called "Mass Law", that describes exactly what you want. It goes like this:
TL(dB)= 20log(M) + 20log(f) -47.2
Where:
TL is the Transmission Loss (how much isolation you get)
M is the surface density of the panel (mass per unit area in kg/m² ), and
F is the center frequency of a one-third-octave measurement band.
You can plug in the numbers, and get the answers you want.
I did that for you, and to get 20 dB reduction at 150 Hz, you would need a panel that has a surface density of about 15.1 kilograms per square meter:
20log(15.1) + 20log(150) -47.2 = 20.001 dB
So, if you wanted to use plywood to do that, plywood has an absolute density of roughly 560 kg/m3, so your panel would need to be 15.1/560 = 26mm thick. That's a little more than one inch. Rather thick. If you wanted to use a thinner panel, it would need to be more dense. MDF is around 720 kg/m3, so an MDF panel would be 15.1/720=21mm thick. That's about 13/16", or a bit thicker than 3/4". If you wanted it thinner still, you could try glass, which is about 2500 kg/m3, so you could use glass just 6mm thick. That's about 1/4".
Those are all options that would give you 20 dB of isolation at 150 Hz. It always gets better as you go higher up in frequency. And you can predict that too: If you look closely at the equation, you'll see that isolation increases at the rate of 6 dB per octave, so you'd be fine for higher frequencies. However, that equation is theoretical for perfect materials in a perfect world: In reality, the rate is more like 4 to 5 dB per octave for real materials in the real world. Let's call it 5 dB/octave to make the math simple. Thus, at 300 Hz you would get 25 dB, at 600 Hz you'd get 30 dB, at 1.2 kHz = 35 dB, at 2.4 kHz, 40 dB, etc. Total overall isolation would be a bit more than 30 dB.
Acoustic theory is a wonderful thing! It saves you buying materials that won't work, since it can predict what would happen if you did. Based on this, to get isolation of 20 dB, from 150 Hz upwards, you just need to build a box from one inch thick plywood, and seal it air-tight all around. In fact, you could cut out a section of the 1" plywood on one of the sides and replace that with 1/4" thick glass so you can see inside. But since you need this for vocals, the box would have to be big enough to fit a person inside comfortably, along with a chair, mic, mic stand, and music stand, as well as having acoustic treatment inside to make it sound good, and a ventilation system, so the singer can breathe and stay comfortable during the session.... In other words, you need a vocal booth!
OK, so that's a rather round-about way to describe to you why recording studios have vocal booths: because it's the only way to do what you want. They already tried every possible alternative under the sun, just like you did, and they figured out in the end that the only way to get clean vocal tracks and clean drum tracks, is by putting the vocalist in an isolation booth.
There are no short-cuts, unfortunately! If there were, then somebody would have figured it out already, and put a product on the market... But the oly ones who tried that, failed. As you can see form the Sound on Sound article.
Sorry to be the bearer of bad news, but the only way to do what you want is to get the singer out of the live room, and into either the control room, or a vocal booth.
- Stuart -