Can this software create 3D meshes from a MIDI file?

[sebarious70]

I'm trying to create a 3D scene from a midi file. The scene would need to have each note as a cube rectangle, and the translation from the music world into the 3D scene would be basic: 1 bar = 1 meter. So if you have a MIDI note that is 1 bar long, the long side of the rectangular cube would be 1 meter long. The other two dimensions would be 25 cm each. So, something like this:

I'm on the verge when it comes to the other two dimensions for shorter notes, because if I keep 25 cm for all of them, when I do notes that are 1/2, 1/4 and 1/8, it would look like this:

But if I also shorten X and Z more or less proportionally, I think perhaps it would look better:

Regardless, my goal is to get something like this:

into something like this:

Obviously without the flat notes, those are just to show the difference between the flat ones and how the 3D ones should be)

My secretary (that would be the free version of Chat GPT) pointed me to this excellent software when I asked it if there was any program that can generate 3D meshes from MIDI notes. So I started reading the documentation and I found some hope, but first I want to ask here if what I'm trying to achieve is possible with this add-on. I don't even need animation, just static notes, all laid out in a 3D scene.

Now, there's a section of the documentation that I spent a lot of time trying to build exactly as it shows, but I didn't get anywhere near the animation that shows there, or saw any meshes being built: . The only thing I achieved was to move the first cube I created in -Y.

Now, I copied the node tree under "Raw Note Information" as best as I could, but there is one node, the group input all the way to the left that has two inputs, one is a MIDI track, and the other is Width. Well, "Width" is not a name that shows up when I click on the + and type width. In fact, it doesn't appear in anything. Either the documentation is outdated or Blender 4.1.0 has something so it doesn't show? I couldn't say. But other than that, I think all the rest are just like the example in the link above:

So that's all. Basically I want to know if this add-on can be setup in a way that will ingest a MIDI file (and it won't be a super complicated one, just one track), and turn that into a 3D scene, and if so, if this is more or less the node setup I need, or if I need to go another route.

[OmarEmaraDev]

The Width input is just a float input whose name is Width. You can definitely create what you want using this software, I can provide an example file if you couldn't get it to work eventually.

[sebarious70]

Good, at least now I know it's possible. Now, what's the learning curve? Because what I need to do can be done manually by setting the viewport to top view and putting a long wide screenshot of the section of the song that I want to do this for as a background, then grab the 1 bar note, duplicate several times, then the same thing with the other lengths, and finally placing them to match the notes. Time consuming, but not terribly so.

On the other hand, if the automation nodes way means spending days learning the stuff, that's time that I don't have. Would love to, because it looks like a great thing to learn, but I don't have the luxury of time.

Basically the thing I study music, and I have some songs that I would like to upload to YouTube, but with visuals, and I have a waveform that I did in Fusion with a 3rd party plugin, but then I thought about adding a way to visualize parts of the song with a MIDI track in 3D. But this is something I want to upload as soon as possible, so I can't spend a week or two learning this.

How difficult is it to get it to create the meshes from a simple MIDI track, only index 0? I've been trying some things this afternoon but I couldn't find the way to actually generate meshes from the MIDI track.

[OmarEmaraDev]

@sebarious70 I am not not sure about the learning curve, because it likely depends on how familiar are you with Blender, nodes, maths, and so on. But this particular thing you want to do is rather easy, so it is probably worth learning a bit, and we can help you along the process.

I made you a simple example to get you started:

Just change the path to your MIDI file and you should get MIDI notes as meshes. The Get List Element retrieves the first track from the file. Then we get the note info of that track, and compute the note length by subtracting the on time from the off time. The X position of the note is its On time, its Y position is its note number. The X scale of the note is the note length, while its Y and Z scales are fixes, which you may change based on some note info.

midiToMesh.zip

[OmarEmaraDev]

@sebarious70 You can use an Object Instancer node to create an object for each note.

Maybe look into the Copy Modifiers and Object Material Output nodes to achieved what you want.

[sebarious70]

That's awesome! I applied this to the node flow you gave me previously and it works like a charm.

One thing I noticed is that it transfers the material slots from the original cube, which is great. But it doesn't transfer the modifiers. Is there a way to do that?

And the type colors, is there a way to change them? The object black color is really hard to tell agains the dark background, and I'm my 50's, so my eyesight is not the best.

[OmarEmaraDev]

@sebarious70 You can check the Copy Full Object option in the Instancer node to copy the modifiers as well.
What do you mean by Type Colors?

[sebarious70]

Thanks!

I mean, different classes of inputs and outputs have different colors, like matrices are orange, vectors out is blue, and the object to object is black. Since the background is I'm guessing 90% black, it makes it really hard to see.

[OmarEmaraDev]

The colors are not themable unfortunately. But you can change the background color in Blender's theme setting somewhere. Or just choose a different theme that is more contrasty.