Ticket #138: audio-discussion-log.txt

[12:01] *** #wfg: gavin_f @jw_away
[12:01] *** #wfg was created on Sun Aug 06 05:39:41 2006.
[12:01] *** Mode change "+o gavin_f" for channel #wfg by jw_away.
[12:01] jw_away: howdy!
[12:01] *** jw_away is now known as janwas.
[12:01] gavin_f: mornin!
[12:02] gavin_f: I didn't get a response from Damian, so he may not have gotten my email =/
[12:02] janwas: that's too bad, but we can hack it alone :)
[12:03] janwas: first off, did any questions come up while working through the code>?
[12:03] gavin_f: yeah, let me get some stuff pulled up real quick.
[12:03] janwas: awwright.
[12:05] gavin_f: oh, one that comes to mind, where is vSrc defined?
[12:05] janwas: right. snd_mgr.cpp
[12:05] gavin_f: hah, woops, missed that one...
[12:06] janwas: tip: install VisualAssist, then just hit Alt+G when cursor is on an identifier
[12:06] janwas: it'll take you to the spot of its definition.
[12:06] gavin_f: is there a free version of Visaul assist?
[12:07] janwas: free trial:
[12:07] janwas: http://www.wholetomato.com/downloads/VA_X_Setup.exe
[12:07] janwas: it is possible to reset the trial date
[12:07] gavin_f: =)
[12:09] gavin_f: k, got it.
[12:10] gavin_f: cool, so that has a lot of the stuff I was needing to store.
[12:10] janwas: once you've got that installed: what would you like to do next? what is the current status?
[12:10] janwas: i would suggest going through the task list, clearing up questions there, and setting priority
[12:10] gavin_f: can't believe that I didn't see that in there...
[12:10] gavin_f: agreed
[12:10] janwas: :) snd_mgr is quite big.
[12:10] janwas: are you using VC2003 or lateR? code can be hidden (collapsed), which is very useful
[12:11] gavin_f: yeah, I'm using 2k3, haven't had time to get into 2k5 yet, though I have it installed as well.
[12:11] janwas: not a problem, both work.
[12:11] gavin_f: is there any kind of container being used to store vSrc's right now?
[12:12] gavin_f: like a list or queue of them?
[12:12] janwas: simply select a bunch of unnecessary code, hit ctrl+m, ctrl+h, then its hidden
[12:12] janwas: ah, there is a bit of magic going on. VSrc is a "resource". [..]
[12:12] janwas: all resources are stored by the handle manager - there is an array of them, and the handle manager (h_mgr) takes care of finding a free entry, and returning a handle to it
[12:12] janwas: this array has some fixed-size storage space, into which the VSrc is stored.
[12:13] janwas: so basically they are already stored in a container, no need to worry about that.
[12:13] janwas: also:
[12:14] janwas: snd_mgr has a section (list_*) which maintains a list of all active VSrc. this is useful when iterating over all of them (e.g. to recalc their priority)
[12:14] gavin_f: good deal
[12:14] janwas: note that the iteration would be possible by iterating over all handles of type VSrc, but that is a bit ugly and slow
[12:15] gavin_f: because not all the VSrc's stored are going to be sounds, right?
[12:15] gavin_f: rather...
[12:15] janwas: not sure what you mean. each VSrc is a sound that might potentially be played. if a VSrc is at the other end of the map, it will not actually be played.. [..]
[12:15] janwas: but we still pretend that the sound played.
[12:16] gavin_f: yeah, nm my previous comment.
[12:16] janwas: (think of it like this: anything the game might want to play is a VSrc. they are all played, only the ones that are far away are really quiet.)
[12:16] gavin_f: gotcha
[12:17] gavin_f: any problems with adding a data member or two to VSrc>
[12:17] janwas: (now those that are so quiet that you won't hear anything anyway, we don't actually send them to the soundcard at all. this is necessary because the card can usually only play 32 at a time)
[12:17] gavin_f: if needed?
[12:17] janwas: nah, probably not.
[12:17] janwas: bear in mind that VSrc is stored in this fixed-size area, so we can't put 2kb of data in there
[12:17] janwas: but if you make it too big, you will automatically get an error message, so all is well
[12:18] janwas: any idea already as to what would need to be added?
[12:18] gavin_f: not yet, just wanted to make sure if it was needed that it would be possible.
[12:19] janwas: aight.
[12:19] janwas: note that VSrc is initialized to 0. any fields added will have to be set to their desired initial value (if it's not 0) in VSrc_init
[12:19] janwas: (basically the constructor)
[12:19] gavin_f: did you happen to see Damian's response in my tasklist?
[12:19] gavin_f: (understood)
[12:20] janwas: (it may be helpful to read h_mgr.h, see what all the resource/handle business entails)
[12:20] gavin_f: kk
[12:20] janwas: damian's response: yes indeed. all good there
[12:22] janwas: any more questions, or shall we go through that thread?
[12:22] gavin_f: As far as volume control is concerned, I saw the master attenuation, and I was looking at the OpenAL documentation to determine a good way to do seperate volumes such as one for SFX and one for Music, etc.  Any ideas off the top of your head how to implement that?
[12:22] gavin_f: yeah, let's take a look at the task list.
[12:22] janwas: ok, the gain thing:
[12:22] janwas: the value of VSrc.gain is what will actually be sent to the hardware (note:
[12:23] janwas: there is a master gain that is applied to all OpenAL sources, but we can ignore that for the moment)
[12:23] janwas: we simply store an array of gains for the various sound types (VO, sfx, music)
[12:23] janwas: when the game says: i want to play a sound.. it would call snd_load and then snd_play.
[12:24] janwas: in between, we can snd_set_gain.. and we would pass in sound_type_gain (from the array above) TIMES desired_gain_for_this_file
[12:24] janwas: desired_gain.. is taken from the XML file that defines the sound file's properties
[12:25] janwas: it is necessary because sounds are usually stored at full volume, and the desired_gain is what allows us to make a birdsong less loud than a full-on battle
[12:25] janwas: so, no problem there.
[12:25] janwas: ok?
[12:26] gavin_f: ok, so when the player adjusts the SFX volume, we are going to store that(in the array), then when an SFX is played, we reference that information.  I think I got it.
[12:27] gavin_f: Is there something in VSrc.flags that defines the type of sound?
[12:27] janwas: right.
[12:27] janwas: the separate types of gains:
[12:27] janwas: - master (applied automatically by the hardware, no intervention needed)
[12:27] janwas: - desired_gain (per-file; must be passed to snd_set_gain)
[12:27] janwas: - sound_type_gain (per-type; must also be passed to snd_set_gain - multiplied with the above)
[12:28] janwas: nope, VSrc does not know anything about the type. consider that 0ad might have a certain idea of what types of sounds it can play.. but snd_mgr.cpp should be reusable across projects and doesn't know anything about sound types
[12:29] janwas: i was thinking the sounds would be classified by their type on a higher level, i.e. by the game code that decides to play a sound
[12:29] janwas: for instance, we could have the following interface:
[12:29] janwas: playVoiceOver(), playMusic, playSfx etc.
[12:29] gavin_f: that sounds feasable.
[12:29] janwas: OR define an enum SND_TYPE_MUSIC, etc., and then playSoundType(snd_type)
[12:30] gavin_f: let the play function worry about what volume to set.
[12:30] janwas: (the former interface would probably be implemented via the latter)
[12:30] janwas: right.
[12:30] janwas: slight wrinkle: depending on whether it is C++ or JS code that wants to play music..
[12:30] janwas: we may need to change JSI_sound.cpp. that is the glue code that allows JS code to call snd_mgr
[12:31] janwas: it is possible that the above playSfx etc. functions must be callable from JS.
[12:31] janwas: i am not sure who would trigger sword clang sounds - probably the C++ entity code
[12:32] janwas: **** that's an interesting question. in response to what are sounds triggered? when a unit is attacking, does it emit a clang noise every N seconds?
[12:32] janwas: that's over my head, we'll have to ask philip or matei.
[12:32] gavin_f: hrm...yeah, probably several ways we could handle that.  retriggering by the entity, or something else
[12:33] janwas: right. let's ignore that issue for now an assume a playSound(soundGroup) function is called by the entity code.
[12:33] gavin_f: as far as adapting JS code to call one of the c++ play functions, that should be too hard to do right?
[12:33] gavin_f: *shouldn't
[12:33] janwas: *shouldn't* ?
[12:33] janwas: yep.
[12:34] janwas: it's a bit of black magic, what all the JS interpreter wants, but once you have seen an example (e.g. the current JSI_sound), just replicate that and it's fine
[12:34] gavin_f: kk
[12:34] janwas: back to soundGroup. that would be task #1: implement logic to play a random track from it (trying to avoid repetition)
[12:35] janwas: example of soundGroup is "sword clangs"; we will have several of those
[12:35] gavin_f: right on.  I'll need to take a look at where all the sounds are stored.
[12:36] gavin_f: that is, in their folders within the project.
[12:36] janwas: yes. for "soundgroup", see here: http://www.wildfiregames.com/forum/index.php?showtopic=8773
[12:36] janwas: starting here: then to audio_group.xml for soundfile(s) and adjustment properties
[12:36] janwas: on your HD: data\mods\official\audio
[12:36] janwas: from in-game: "audio/"
[12:37] janwas: you understand the distinction, i.e. how the filesystem accesses files?
[12:37] gavin_f: yeah
[12:37] janwas: awwright.
[12:38] janwas: i would consider this soundGroup functionality as laid out in the thread to be important.
[12:38] janwas: (and one of the first things to implement; will have to see if anything else needs to be done first, among the other tasks)
[12:41] gavin_f: right on, I was looking at a scheme to load the music tracks for non-standard randomization.  for example, I can look at audio/music and grab everything that says germanic_peace_*.ogg and load those fairly easily.
[12:42] gavin_f: then just enumerate them and compare when picking the next track to make sure it hasn't been played already.
[12:42] janwas: ok, right.
[12:42] janwas: another idea:
[12:43] janwas: you could enumerate them (vfs_enum), or ps/VfsUtil.h, and then generate a permutation of the files
[12:43] janwas: std:: shuffle, IIRC
[12:43] janwas: no, random_shuffle
[12:44] janwas: that way, you could generate a list of the next few songs to play; when one is played, you remove it from the front of the list. when list is empty, fill it again and re-shuffle
[12:45] janwas: random_shuffle has the big advantage of avoiding numerous pitfalls - it is easy to get shuffling wrong, e.g. not yielding uniform distribution among the N! permutations
[12:46] gavin_f: gotcha, looking at it in MSDN right now
[12:46] janwas: aight.
[12:46] janwas: the list would probably be maintained by / stored in a SoundGroup objcet, which is loaded from the audio_group.xml file as detailed by damian.
[12:48] gavin_f: k, that doesn't look too bad.
[12:49] janwas: right. we will also need a list of soundGroups:
[12:49] gavin_f: do you know if Damian intends on using one audio_group.xml to define each group of sounds, or is it going to be more of a single master list?
[12:50] janwas: (background) playSound(soundGroup) is what the game calls, but we don't want the game to explicitly load all soundGroups
[12:50] janwas: it would be nicest if a soundGroupManager were to remember which soundGroups are currently in memory (have been loaded) and load them if not
[12:50] janwas: this manager provides access to the relevant group, loading it if necessary
[12:50] gavin_f: k, that sounds feasable, if nothing else just store the name of the xml file, and check before loading again.
[12:50] janwas: every group is in memory only once; it keeps that list of tracks (in shuffled order)
[12:51] janwas: there should be several audio_groups. it is possible that sword clangs should be repeated, but e.g. OOF sounds could be played in any order (allowing repetitions)
[12:51] gavin_f: OOF?
[12:51] gavin_f: nm
[12:51] janwas: just a stupid example, but there are other properties in audio_group.xml that would make it worthwhile to support several
[12:52] janwas: hehe :)
[12:52] gavin_f: oof!
[12:52] janwas: yep.
[12:52] janwas: yeah - "storing" xml file could be a std::map or STL_HASH_MAP of filename -> the soundGroup object in memory
[12:53] gavin_f: any estimates on how many different sound groups we are going to want to have loaded at any given time?
[12:54] janwas: good q. i imagine each civ will have a different OOF soundset, because the XML file references the exact filenames.
[12:55] gavin_f: yeah, might be a good question for Damian.
[12:55] janwas: hm, then again, we could prepend the entity's civ name
[12:55] janwas: it's an implementation detail, i guess - no need to bother damian. we could probably get away with the prepending thing
[12:56] janwas: that'd be good, because one group for each civ and actual group would be wasteful
[12:56] gavin_f: prepending the civ name would be adding the civ name to the front of the file name?
[12:56] janwas: estimate of #groups: unit types = gaia, foot, cav, ship, villager. each has 3..5 sounds they can make => call it < 50 groups
[12:56] janwas: yep.
[12:57] janwas: the voice files are grouped in audio/voice/hellenes, so that'd work.
[12:58] janwas: ok. moving right along: the plan covers playing random sounds from a group. that takes us mostly there already! the remaining features needed are:
[12:59] janwas: priority, intensity, soundClasses
[12:59] janwas: all you need to do with priority is define a set of constants that the engine's playSound function would use. damian has given a good list in the progress report thread.
[12:59] gavin_f: yeah, that seems pretty straight forward
[13:00] janwas: intensity: we will need a means of observing the sound requests that the engine pounds out. if, within (say) 100ms, a LOT of sword clangs are produced, you can cancel the sword clang sounds and replace them with one "big battle" clip
[13:00] janwas: this is something that should be doable separately and not important for now, so it can be tabled for later
[13:00] gavin_f: kk
[13:01] gavin_f: will keep that in mind as I'm going.
[13:01] janwas: righto (y)
[13:01] janwas: soundClasses: remember the array of sound types? there ar eprobably a few other properties that would be useful for soundClases. again, damian has a list in audio_class.xml
[13:02] janwas: consider these values a template or default for the each sound that belongs to the sound class
[13:02] janwas: sound class could be voiceover, music, etc. as mentioned
[13:03] janwas: first, as to what values are necessary: i figure only gain and default pri are needed for now. the rest can be added later (easily) if they turn out to be necessary
[13:03] gavin_f: right on, I think I'll get back on top of that class I was making to store soundClasses
[13:03] janwas: right.
[13:03] janwas: the soundClass will store the defaults, and whenever playing a sound, it will apply those defaults to the vsrc properties
[13:04] gavin_f: Basically store those values in addition to a VSrc.
[13:04] gavin_f: then apply them when a sound is played
[13:04] janwas: yeah. there is one global copy of the defaults for VOICEOVER, and then the actual VO Vsrc object has the actual values as well
[13:04] janwas: exactly.
[13:04] janwas: there is one wrinkle: gain will need to be *combined* with the sound file's gain (instead of the sound file's gain merely replacing the default)
[13:04] gavin_f: the global copy would be the one adjusted if say, the user adjusted volume control in a menu?
[13:05] janwas: this is to allow the individual sound type volume control thingyh.
[13:05] janwas: exactly.
[13:05] janwas: hohum, one problem: after changing the menu value,
[13:05] janwas: (one sec, neghbor here)
[13:05] gavin_f: yeah, master attenuation should always change the volume of everything (if it gets set to 0, no sounds are heard even if they have a gain of 1 set elsewhere)
[13:06] gavin_f: np
[13:07] janwas: yes - master attenuation is already covered
[13:07] janwas: but what about a music track that plays for several minutes? ideally, we would see the changes immediately after altering music volume in the settings
[13:07] janwas: without further work, our scheme will result in changes only being picked up the next time such a sound plays (and the defaults are applied to it)
[13:07] janwas: is that bad?
[13:07] gavin_f: right, changing a volume property will need ot have an immediate effect.
[13:09] janwas: ok, here's what we could do
[13:09] gavin_f: that's probably bad =/  In FMOD there was a somewhat convaluted way of handling this.  I'm thinking of making my own little OpenAL wrapper to play around with how volume controls and stuff might work.
[13:09] janwas: iterate over all sound objects, and recalculate their volume (first, apply the sound class default, then multiply on the sound file's gain)
[13:09] janwas: that'll be a bit slow, since it involves reading in all the per-sound XMLs
[13:10] gavin_f: that would work, but if we have a bunch of sounds it may get to be noticably cumbersome. not too sure
[13:10] janwas: yeah? how did FMOD do it?
[13:10] janwas: in openal, you can change the gain (snd_set_gain) at any time
[13:10] janwas: true.
[13:10] janwas: here's an idea: when loading the sound, remember it's XML file's gain and store it in Vsrc.
[13:10] janwas: (not only the *current* gain - also remember original gain)
[13:10] janwas: then simply use that instead of reexamining the XML file
[13:12] gavin_f: yeah, that doesn't sound like a bad idea.
[13:13] janwas: cool. should be quite easy to do, too
[13:13] janwas: ok, let's think back on the complete chain of things.
[13:14] janwas: each entity is going to have to know what soundClass it triggers and the soundGroup
[13:14] janwas: e.g.: a soldier fighting does playSound(SFX, clang_group)
[13:15] janwas: that dispatches to soundGroupMgr, which loads the group if necessary. inside the group's getNextTrack function (<- pick your own names, these suck)
[13:15] janwas: it checks if the group is set up to disallow repeats  or just choose a random track
[13:15] gavin_f: =)
[13:15] janwas: if disallowing repeats, there is a list of tracks in order in which they should be played. if this list is empty (as it is initially), then retrieve the list of filenames from XML and random_shuffle them
[13:16] janwas: we then have a filename.
[13:16] janwas: that is snd_loaded, getting a handle
[13:16] gavin_f: I'm with you so far
[13:16] janwas: we then look at the soundClass, retrieve the OpenAL property defaults, and snd_set_* them
[13:16] janwas: :)
[13:16] janwas: (one thing i forgot: playSound probably should get a priority parameter as well)
[13:17] janwas: (and snd_mgr may have to be modified a little tiny bit, so that "static priority" will be set from that value)
[13:17] gavin_f: yeah, should be easy to include though.
[13:17] janwas: yeah; there already is concept of static_pri, im just not sure if it's actually set
[13:17] janwas: so: once the properties are set, the sound is ready to be snd_play-ed
[13:18] janwas: and then our dudes will emit noise!
[13:18] gavin_f: woot!
[13:18] janwas: :) plan sounds pretty good
[13:18] janwas: so, any questions on the roadmap?
[13:18] gavin_f: I think it's highly do-able.
[13:18] janwas: righto! (y)
[13:19] gavin_f: just to make sure I have the order of priority straight:
[13:19] gavin_f: Non-standard randomization of music tracks
[13:19] gavin_f: loading soundGroups
[13:19] gavin_f: randomization of unit sounds
[13:19] gavin_f: then volume controls?
[13:20] janwas: i would do soundGroup first, because nothing works until that works
[13:20] janwas: (that, and the soundGroupManager)
[13:20] gavin_f: k
[13:20] janwas: then, we can go on to add the randomization feature; until then, just play the filelist in the order of appearance in audio_group.xml
[13:21] janwas: (however, that's really splitting hairs; the randomization feature shouldn't take too long, so both could be implemented in one step without resorting to a placeholder implementation)
[13:21] gavin_f: right on.
[13:21] gavin_f: yeah, randomization should be rediculously easy
[13:22] janwas: :)
[13:22] gavin_f: I think the monster that was stopping my progress before was me trying to reinvent the wheel.  A lot of the stuff I was worried about handling has been taken care of already.
[13:23] janwas: ok, i'm glad we got this all mapped out. indeed it is easier to discuss this than try to come up with a plan alone
[13:23] gavin_f: very true.
[13:23] janwas: yep, there is some groundwork.
[13:24] gavin_f: anything else we need to cover at this point?  I'm out of questions for the time being.
[13:24] janwas: ok, i don't think so.
[13:24] janwas: let me know if you have any more or encounter problems.
[13:25] gavin_f: yeah I will now that I'm over the initial "what are they going to think if I ask this?" fear.
[13:25] gavin_f: kinda silly, I know.
[13:25] janwas: huh? that's a good thing, then - no worries, just ask!
[13:25] gavin_f: will do.  Thanks again for all your help Jan!!
[13:26] gavin_f: I'll update my tasklist and then shoot a quick email to Damian to have him look it over.
[13:27] janwas: ok!
[13:27] janwas: probably the best thing to do would be to write up a quick summary of the roadmap into a trac ticket
[13:27] janwas: not like "do everything" but smaller steps that you can tick off upon every commit
[13:27] janwas: (glad to :) )
[13:28] janwas: it would also be nice to upload this log to the wfg server (your account) and link to it from your progress report (or in whichever thread you would like to post)
[13:28] gavin_f: right on.  I'll get started on that then, and if I run into anything I'l shoot you an email.
[13:28] janwas: cool! (y)
[13:29] janwas: i'll head out, grab some dinner.
[13:29] janwas: see ya!
[13:29] gavin_f: later!