I think we do need some "common lujvo". The gismu themselves (individually)
are not adequate for a basic vocabulary. At the very least, tanru are
needed, and that's why lujvo are supposed to be coined -- to prevent adding
gismu *ad hoc* and *ad nauseam*.
(Sure, there's a case to be made for more gismu. Maybe about fifty. Maybe.
And not desperately.)
"Common" lujvo would have common sets of place structures, would be more
easily learned as language patterns, and would be easier to extend or
modify, requiring the dynamic skill you write about. The set of lujvo
composed of a SE lujvo plus the rafsi -- such as selkla, terkla, velkla, and
xemkla, from the places of klama -- is large, and easy to make "on the fly",
and extends the vocabulary to over 3000 brivla.
Using zma or mau (zmadu) as a "suffix" is like using "-er" in English to
augment something: bigger, greener, happier, etc. It's a common usage and
the place structure isn't too hard to figure out.
In the same way, -cau (claxu) is used for "-less", and -ske (saske) tends to
be used for "-ology".
So, what we need is not a set of cast-in-iron lujvo, but a set of both
usable lujvo *and lujvo-making patterns* that most Lojbanists can agree on,
that come from actual Lojban use and not forcing glico into a lojbo mold.
Since you are using lojban a lot in your conversation sessions, it would be
interesting if there was some way of keeping track of the tanru and/or lujvo
you use. If you found a certain one being used periodically, it would be a
good candidate for a lujvo. Recording and "deciphering" conversations may be
difficult to do, but it is one way of establishing what's "common" and what
isn't.