Heat Treating 6061-T4 without quenching

You have some things right; some wrong. "Is it possible to just age the frame after welded let's say 6 hours at 350 degrees?" This is the common practice. It is stroonger than "as welded" though not as strong as resolution treated and aged which is expensive and warps parts. 6061 has a pretty sluggish quench and even has a fair response quenched in air. The material surrounding the weld tends to act as a quench as well. So you get kind of a low quality quech in the heat affected zone. Remember the weld metal isn't 6061 so you get nearly nothing there regardless. Therefore reaging stengthens the heat affected zone and does a good stress relief as well. The aging cycle is 350 degF for 8-10 hours not 6.
"Or is that going to be worse, since I am re-aging a T6 tube and will be in fact over-aging it and make it very brittle." The second age has very little effect on the basic tube. There is no appreciable overaging. Remember "overaging" is not "over hardening". T6 is peak strength. Overaging, which doesn't apply here, would reduce strength and brittleness not increase it.
"What would happen with the actual weldment and annealed area?" It isn't annealed. It will have an aging response from the as-welded; not a good as resolution treated, but there is a response.
"I am trying to find the easiest way to recover some strength at the joints, without making it worse than not doing anything at all.". Not sure what you are going for here. Welded 6061 using, say, 4043 filler, will never be a strong as the basic 6061 tubing. No combination of welding process, filler, and heat treatment will get you to unwelded properties. All processes have effects on performance; welding has one on the biggest effects. As-welded properties are the worst. T4, welded and aged to T6 only, is better. T4 welded then solution treated and aged is the best. This is also the order of increasing costs. I prefer T4 welded then aged to T6 as a good balance between cost and performance.
That should help. Now you have some design, stress, and testing work to do.