Wire Sound Synthesis!

[Nikita]

Please kindly move this thread to an appropriate forum if this isn't the one, as the subject is as much on math as it is on wire building.

I am looking to create a device that can generate the closest possible replica of a sound file (say, a wav file) converted into an array of ram-like wire memory. Imagine I already have the file in memory. How would the device function?

Perhaps we would have to use complex math and Furier transforms, which I don't understand, but they can be used for a number of things, like:
a)breakdown of music into volumes of harmonics (waveforms with commonly divisible frequency) which then feed into sound emitters of said harmonics
b)frequency modulation, which makes a graph of frequency of a single waveform, that we can feed into sound emitters instead.

Another possible idea that I had is have a sample of a graph's "hill" (sinewave's half-period) and a sample of the opposite half-period, calculate the sound as an astronomic number of such half-periods that overlap, and just feed to a dozen of sound emitters (both volume and frequency).

I don't know how it should be done. Perhaps there are a few great matematicians or sound specialists here.

But either way I'm making some sound synth contraption even if it's no better than a sampled piano.

[Jeremydeath]

An interesting idea, but unfortunately not really possible with the source sound engine. The sound just take too long to start/stop to be practical.

The closest thing that I've got to anything like this is:
[highlight=E2]@name Ring Tone Transfer Language E2
@inputs
@outputs Name:string Compleat
@persist DefDuration DefOctave DefTempo
@persist NotesList:array CurrentNote NoteCount
@persist ThritySecondNoteTime
@trigger all

if(first())
{
#SongInfo = "HauntedHouse: d=4,o=5,b=108: 2a, 2e, 2d#, 2b, 2a, 2e, 2d, 2a#4, 2e., e, 1f4, 1a4, 1d#, 2e., d, 2c., b4, 1a4, 1p, 2a4, 2e, 2d#, 2b4, 2a4, 2c, 2d, 2a#4, 2e., e, 1f4, 1a4, 1d#, 2e., d, 2c., b4, 1a4"
#SongInfo = "fifth:d=4,o=5,b=63:8P,8G5,8G5,8G5,2D#5"
SongInfo = "TocattaFugue:d=32,o=5,b=100:a#.,g#.,2a#,g#,f#,f,d #.,4d.,2d#,a#.,g#.,2a#,8f,8f#,8d,2d#,8d,8f,8g#,8b, 8d6,4f6,4g#.,4f.,1g"
#SongInfo = "The shoop shoop song: d=4,o=5,b=120:g,g,g,g,f,8f,8d#,8f,8d#,c,g,8g,8g,g, 8g,8g,b,8g,g.,8e,8d,8f,e,d."
#SongInfo = "Scatman:d=4,o=5,b=200:8b,16b,32p,8b,16b,32p,8b,2d 6,16p,16c#.6,16p.,8d6,16p,16c#6,8b,16p,8f#,2p.,16c #6,8p,16d.6,16p.,16c#6,16b,8p,8f#,2p,32p,2d6,16p,1 6c#6,8p,16d.6,16p.,16c#6,16a.,16p.,8e,2p.,16c#6,8p ,16d.6,16p.,16c#6,16b,8p,8b,16b,32p,8b,16b,32p,8b, 2d6,16p,16c#.6,16p.,8d6,16p,16c#6,8b,16p,8f#,2p.,1 6c#6,8p,16d.6,16p.,16c#6,16b,8p,8f#,2p,32p,2d6,16p ,16c#6,8p,16d.6,16p.,16c#6,16a.,16p.,8e,2p.,16c#6, 8p,16d.6,16p.,16c#6,16a,8p,8e,2p,32p,16f#.6,16p.,1 6b.,16p."
#SongInfo = "Deutschlandlied:d=4,o=5,b=160:2g,8a,b,a,c6,b,8a,8 f#,g,e6,d6,c6,b,a,8b,8g,2d6,2g,8a,b,a,c6,b,8a,8f#, g,e6,d6,c6,b,a,8b,8g,2d6,a,b,8a,8f#,d,c6,b,8a,8f#, d,d6,c6,2b,8b,c#6,8c#6,8d6,2d6,2g6,8f#6,8f#6,8e6,d 6,2e6,8d6,8d6,8c6,b,2a,16b,16c6,8d6,8e6,8c6,8a,2g, 8b,8a,2g"
#SongInfo = "Macarena:d=4,o=5,b=180:f,8f,8f,f,8f,8f,8f,8f,8f,8 f,8f,8a,8c,8c,f,8f,8f,f,8f,8f,8f,8f,8f,8f,8d,8c,p, f,8f,8f,f,8f,8f,8f,8f,8f,8f,8f,8a,p,2c6.,a,8c6,8a, 8f,p,2p"
#SongInfo = "ScoobyDoo:d=4,o=5,b=160:8e6,8e6,8d6,8d6,2c6,8d6,e 6,2a,8a,b,g,e6,8d6,c6,8d6,2e6,p,8e6,8e6,8d6,8d6,2c 6,8d6,f6,2a,8a,b,g,e6,8d6,2c6"
#SongInfo = "PinkPanther:d=4,o=5,b=160:8d#,8e,2p,8f#,8g,2p,8d# ,8e,16p,8f#,8g,16p,8c6,8b,16p,8d#,8e,16p,8b,2a#,2p ,16a,16g,16e,16d,2e"
#SongInfo = "Barbie girl:d=4,o=4,b=125:8g#,8e,8g#,8c#6,a,p,8f#,8d#,8f# ,8b,g#,8f#,8e,p,8e,8c#,f#,c#,p,8f#,8e,g#,f#,8g#,8e ,8g#,8c#6,a,p,8f#,8d#,8f#,8b,g#,8f#,8e,p,8e,8c#,f# ,c#,p,8f#,8e,g#,f#,8g#,8e,8g#,8c#6,a,p,8f#,8d#,8f# ,8b,g#,8f#,8e,p,8e,8c#,f#,c#,p,8f#,8e,g#,f#"
#SongInfo = "Argentina:d=4,o=4,b=70:8e.4,8e4,8e4,8e.4,8f4,8g4, 8a4,g4,8p,8g4,8a4,8a4,8g4,c,g4,8f4,e.4,8p,8e4,8f4, 8g4,8d4,d4,8d4,8e4,8f4,c4,16p,8c4,8d4,8c4,8e4,g4,1 6p,8g4,8g4,8a4,c,16p"
#SongInfo = "FunkyTown:d=4,o=4,b=125:8c6,8c6,8a#5,8c6,8p,8g5,8 p,8g5,8c6,8f6,8e6,8c6,2p,8c6,8c6,8a#5,8c6,8p,8g5,8 p,8g5,8c6,8f6,8e6,8c6"
#SongInfo = "National Anthem:d=4,o=5,b=140:g5,g5,a5,f#5.,8g5,a5,b5,b5,c6 ,b6.,8a5,g5,a5,g5,f#5,g5"
#SongInfo = "Greensleaves:d=4,o=5,b=140:g,2a#,c6,d6.,8d#6,d6,2 c6,a,f.,8g,a,2a#,g,g.,8f,g,2a,f,2d,g,2a#,c6,d6.,8e 6,d6,2c6,a,f.,8g,a,a#.,8a,g,f#.,8e,f#,2g"
#SongInfo = "Hitchcoc:d=4,o=5,b=200:16c,16p,16f4,8p,8f,32g,32p ,16f,32p,16e,32p,16d,32p,16e,8p,16f,32p,16g,8p.,16 c,16p,16f4,8p,8f,32g,32p,16f,32p,16e,32p,16d,32p,1 6e,8p,16f,32p,16g,8p.,16c,16p,16f4,8p,16g#,32p,8c6 ,16p,16a#,32p,16g#,8p,16c6,32p,8d#6,16p,16c#6,32p, 16c6,8p,16d#6,32p,8g6,16p,16f6,32p,16e6,32p,16c#6, 32p,16c6,32p,16a#,32p,16g#,32p,16g,32p,8f4"
#SongInfo = "missathing:d=4,o=5,b=125:2p,16a,16p,16a,16p,8a.,1 6p,a,16g,16p,2g,16p,p,8p,16g,16p,16g,16p,16g,8g.,1 6p,c6,16a#,16p,a,8g,f,g,8d,8f.,16p,16f,16p,16c,8c, 16p,a,8g,16f,16p,8f,16p,16c,16p,g,f"
#SongInfo = "hipsdontlie:d=4,o=5,b=100:c6,8a#,16c6,16a#,16p,16 a#,16a#,16a#,8g,8a#,8g,8g,16f,16f,16g,16f,16p,16d# ,8c,8p,16c,16c,8d#,8c,8p,16p,16c,8d#,8c,8p,16c,16c ,8d#,16c,16c,16d#,8c,16c,8d#,8c,p,16p,16c6,16c6,16 c6,8a#,16c6,16a#,16p,16a#,16a#,16a#,8g,8a#,8g,8g,1 6f,16f,16g,16f,16p,16d#,8c,8p,16c,16c,8d#,8c,8p,16 p,16c,8d#,8c,8p,16c,16c,8d#,16c,16c,16d#,8c,16c,8d #,8c "


#================= Get Raw Data ===============
SongData = SongInfo:explode(":")

Part = 1
#get the name
Name = SongData:string(Part)
Part++

#get the raw settings
Settings = SongData:string(Part):lower()
Part++
if(!Settings:find("="))
{
Settings = SongData:string(Part):lower()
Part++
}

#get the raw notes
NotesString = SongData:string(Part):lower()
Part++

#================= Translate Settings ===============
SettingInfo = Settings:trim():explode(",")
N = 1
#get the first setting info number
if(SettingInfo:string(N):trim():lower():left(1) == "d")
{
DefDuration = SettingInfo:string(N):trim():lower():explode("="): string(2):toNumber()
}
elseif(SettingInfo:string(N):trim():lower():left(1 ) == "o")
{
DefOctave = SettingInfo:string(N):trim():lower():explode("="): string(2):toNumber()
}
elseif(SettingInfo:string(N):trim():lower():left(1 ) == "b")
{
DefTempo = SettingInfo:string(N):trim():lower():explode("="): string(2):toNumber()
}
N++

#get the second setting info number
if(SettingInfo:string(N):trim():lower():left(1) == "d")
{
DefDuration = SettingInfo:string(N):trim():lower():explode("="): string(2):toNumber()
}
elseif(SettingInfo:string(N):trim():lower():left(1 ) == "o")
{
DefOctave = SettingInfo:string(N):trim():lower():explode("="): string(2):toNumber()
}
elseif(SettingInfo:string(N):trim():lower():left(1 ) == "b")
{
DefTempo = SettingInfo:string(N):trim():lower():explode("="): string(2):toNumber()
}
N++

#get the third setting info number
if(SettingInfo:string(N):trim():lower():left(1) == "d")
{
DefDuration = SettingInfo:string(N):trim():lower():explode("="): string(2):toNumber()
}
elseif(SettingInfo:string(N):trim():lower():left(1 ) == "o")
{
DefOctave = SettingInfo:string(N):trim():lower():explode("="): string(2):toNumber()
}
elseif(SettingInfo:string(N):trim():lower():left(1 ) == "b")
{
DefTempo = SettingInfo:string(N):trim():lower():explode("="): string(2):toNumber()
}

#================= Get Notes ===============
NotesList = NotesString:explode(",")
NoteCount = NotesList:count()
CurrentNote = 1

#================ Setup Time ================
ThritySecondNoteTime = 60000/(DefTempo*8)

interval(10)

exit()
}


if(clk() & CurrentNote <= NoteCount)
{
#================== Start Note Decoding =================
Note = NotesList:string(CurrentNote):trim():lower()

NoteLength = ThritySecondNoteTime*8*(4/DefDuration)
NoteLetter = "a"
NoteOctave = DefOctave

#check for duration identifier
if(Note:index(1):toNumber() != 0)
{
Index = 1
#get the note length
if(Note:index(1):toNumber() != 0 & Note:index(2):toNumber() != 0)
{
NoteLength = ThritySecondNoteTime*8*(4/Note:sub(Index,Index+1):toNumber())
Index += 2
}
else
{
NoteLength = ThritySecondNoteTime*8*(4/Note:index(Index):toNumber())
Index++
}
if(Note:right(1) == ".")
{
NoteLength *= 1.5
}

#get the note letter
if(Note:index(Index+1) == "#")
{
NoteLetter = Note:sub(Index,Index+1)
Index += 2
}
else
{
NoteLetter = Note:index(Index)
Index++
}

#get the octave
if(Note:index(Index) == "" | Note:index(Index) == "." )
{
NoteOctave = DefOctave
}
else
{
NoteOctave = Note:index(Index):toNumber()
}

}
else
{
NoteLength = ThritySecondNoteTime*8*(4/DefDuration)

Index = 1
#get the note letter
if(Note:index(Index+1) == "#")
{
NoteLetter = Note:sub(Index,Index+1)
Index += 2
}
else
{
NoteLetter = Note:index(Index)
Index++
}

#get the octave
if(Note:index(Index) == "" | Note:index(Index) == "." )
{
NoteOctave = DefOctave
}
else
{
NoteOctave = Note:index(Index):toNumber()
}
}


#============ Set The Sound Pitch ==============
NotePitchNum = 1

if(NoteLetter == "c"){
NotePitchNum = 1
}
elseif(NoteLetter == "c#"){
NotePitchNum = 2
}
elseif(NoteLetter == "d"){
NotePitchNum = 3
}
elseif(NoteLetter == "d#"){
NotePitchNum = 4
}
elseif(NoteLetter == "e"){
NotePitchNum = 5
}
elseif(NoteLetter == "f"){
NotePitchNum = 6
}
elseif(NoteLetter == "f#"){
NotePitchNum = 7
}
elseif(NoteLetter == "g"){
NotePitchNum = 8
}
elseif(NoteLetter == "g#"){
NotePitchNum = 9
}
elseif(NoteLetter == "a"){
NotePitchNum = 10
}
elseif(NoteLetter == "a#"){
NotePitchNum = 11
}
elseif(NoteLetter == "b"){
NotePitchNum = 12
}
elseif(NoteLetter == "p"){
#the pause note
CurrentNote++
interval(NoteLength)
exit()
}

NoteFrequency = 261.62*(2^(NoteOctave-5))*(2^(1/12))^(NotePitchNum-1)
Pitch = (5*NoteFrequency)/22
soundPlay(entity():id()+CurrentNote,(NoteLength/1000),"synth/tri.wav")
soundPitch(entity():id()+CurrentNote,Pitch)

CurrentNote++
Compleat = ((CurrentNote-1)/NoteCount)*100

interval(NoteLength)
}
elseif(CurrentNote > NoteCount)
{
CurrentNote = 1
interval(5000)
}
[/highlight]

[Syranide]

Yeah, it all comes down to the fact that the frequency of sound BY FAR exceeds that of Lua in gmod, not only in-terms of performance but also in-terms of that it only executes between frames, meaning you can only change it about every 1/60th of a second (60Hz). And sound is usually 22KHz or 44KHz.

You could probably knock up some fairly impressive voice-synthesis, but it would be primitive and lots and lots of hard work, but would be damned impressive :P

As for recreating music, I'd say it's next to impossible to create something even remotely accurate (or even enough to actually be able to tell what you're actually playing).

[Nikita]

Pretty cool stuff there. The songs are easily recognisable (if you read the names lol). But you can get better quality of you can have two or more sound sources overlap - like on a piano.

I don't share your pessimism about this :P I think that we don't even need to execute that often, though more often is probably better quality. What actually happens every tick is not sound being made, but sound that is already streamed being changed. I believe that if we carefully construct a wire math apparatus to figure out what change of an array of sound emmitter's volume and pitch best corresponds to the change in the source music wave. Sure, that's not precise, but to human ear similarity would be obvious.
I'm not even mentioning that we could use E2 high-speed hackery if we want over 60 Hz.

Though if Jeremy is right about source sound engine limitations, we would be greatly limited. But still, what is in our powers, is still better than anything made and good enough to serve a practical purpose! No?

[Nikita]

Possible solution: Take a wav file with sampling frequency of 44Hz. Convert it into an array of numbers. Make Sound Emitter with sound synth/sine_440 and set it's relative pitch so that the frequency would be exactly 44Hz. Now, the volume needs to be changed 44 times a second, I.e. 1 sample = 1 volume = 1 wave period.
This makes a device which can almost be used to directly set the real speaker's voltage! Yay!

Either this will work perfectly or this will not work at all. Your opinion?
Edit: My opinion is that it will not work at all.

2010 jan 21: trying method A.

[Nikita]

Bump! I am working on this again.

I figured out how to do that. Here is what I will do:


- A spectrogram of a human saying "nineteenth century"

A spectrogram is a 3d graph of volume versus frequency and time. If you break it into a number of horizontal lines, the color would be the volume that a sound emitter of given frequency has to be at each point of time. So, my job would be to simply follow Wikipedia instructions on building a spectrogram, which involve short-time furier transform for each tick. Let's see if I can play a wav song... Oh, first I have to find a way to read past the first 1024 characters in a txt file. Or whatever the limit is.

By the way, is there a way to change phase of a played sound, i.e. make it instantly start at some offset?

[Maxaxle]

Ohhh yessss...
*nerdgasm*

But seriously, this would be EPIC ingame. No more needing to download .wav or .mp3 files, just put together a 5,000-line segment of code and your 30-second audio file is complete!

[ztrain]

hi, if you have a song that is a MIDI you can use this to make it into an E2

midi2e2 Web Interface


also, i have seen a voice synthesis done with the HEV sounds... just throwing out ideas....

[Nikita]

Well... I think I've done all I could do myself. Me+Fourier transforms = monkey + glasses.

I think everything works except for Fast Hartley Transform that I used. I don't have a clue what's going on so obviously I coded it wrong.

Here is what I've done so far:
APERTURE SCIENCE EXPERIMENTAL PROTOTYPE HARMONIC RECONSTRUCTOR
[Clicky]
WHICH IS SO VERY BROKEN AND DOES NOTHING


This archive contains:
1) Wav sound file "Lexx.wav" 43 seconds, 22050 sample rate, 8 bit depth.
2) Program "Spectrograph.exe" (use from command propt)
3) Source code for said program.
4) Output generated for this version.
5) Adv.Dupe containing a contraption for testing.
6) E2's used in the dupe in case they break.

How to test it:
1) Put the Spectrograph.exe in the same folder as whatever wav file you want.
2) Run it through the command, giving name of the wav as an argument.
3) It will create a file called "TransformedWav.txt" this one you copy to e2files folder in Data.
4) You start garrysmod, and spawn the contraption provided.
5) Wait for leftmost indicator to light up. Then toggle the "Arm" button. Screen will show how many seconds left are done loading. Once the second indicator lights up, you hit the play button. If for some reason something somewhere worked, you should hear something similar to the music file.
6) To adjust settings: the "const - voice" E2 is a very handy little one that acts like a constant chip that you control with your voice. Say /a 50 and it changes output A to 50. There are 8 outputs. A is connected multiply gate which controlls relative pitch, so /a 0.05 should theoretically be correct... but try it as 0.2 if you hear nothing. /b controlls volume (I hope).
7) If something goes wronger than it should, try re-uploading the E2s.
End of manual


Would someone please help me? I'm stuck here forever D:

[Xandaros]

I thought about this the other day and I'll try to get this to work, too...

Maybe we can achieve something.

But first I have to get past the RSO my fav server is causing right now -.-

Xan

Edit: Okay, what I managed to get is, if you know how it is supposed to sound, you are able to recognize it. (But barely)

Notice that volume is from 0 to 1. Just divide mastervolume by 500 and you achieve that :P