You're asking several of the Great Unanswerable Questions of the hobby.
Most of the manufacturers announce their forthcoming products at one or more of several big trade shows in Europe and Japan. Sometimes those products come out during the ensuing year; sometimes they don't.
Tamiya often seems to play its cards close to the vest. Sometimes it releases kits out of the blue. If fact it's not unusual for a kit from any manufacturer to appear without advance notice. (Did anybody know in advance that Merit was going to do a series of WWI aircraft in 1/24? I sure didn't - until FSM published a review of the first one.)
How any manufacturer picks its subjects is a carefully guarded secret. Pleas from the modeling community play a role somewhere in the process, but so do quite a few other factors. The dominating consideration, of course, is: "What will sell?"
The one question you've asked that can be answered definitively by an outsider like me is "Does a newer Tamiya kit mean a better kit?" Answer: No. Some Tamiya kits are reissues (of its own kits or Italeri's), sometimes with modifications and sometimes not. Sometimes the company tells us what the modifications are; sometimes not. A few years ago, Tamiya announced with great fanfare that it was releasing several "new" 1/350 battleships. They turned out to be 35-year-old kits in fancy new boxes, at much higher prices. And I'm still torqued about the allegedly "new" 1/700 USS Yorktown (CV5), 99% of which turned out to be a conglomeration of parts from the 1970s-vintage Enterprise and Hornet, both of which suffer from some pretty severe accuracy problems.
Serious scale modelers (except scratch-builders) have a symbiotic relationship with kit manufacturers - and it's a lopsided one. (We need them more than they need us.) The industry in general is mighty close-lipped in its communications with its customers.
Youth, talent, hard work, and enthusiasm are no match for old age and treachery.