A rafsi is not a word. If you say try to say a rafsi by itself, it means something else. For instance, "-coi-" is a rafsi which means "deep", but, if you say "coi" by itself, it means "hello". "coitca" is a lujvo meaning, perhaps, "deep city", but you can't use "coi" and "tca" separately with the same meanings.Isn't an affix something to be "added" to a base word and alter its original grammatical nature (like the English "-er" in "longer" or the Czech "nej-" in " nejhor?í" or the Japanese "sa-" in "samayou")? If it is that rafsi are to be "joined" together to make longer words, mustn't they be certain "root" words themselves from which that resulting longer words' meanings would derive, possessing proper semantic essences even though they are not to be spoken as single words.
If it is a convention to not use rafsi individually, still that doesn't stop them from possessing the nature of base words. You said gismu are "root words" because they are the roots of meaning; rafsi too are the roots of meaning, aren't they? If not, how could we possibly read the meaning of a lujvo which are made from rafsi? rafsi have meanings by themselves, and therefore they are words.