I wouldn't unless the lease terms are very favorable. i.e. Landlord has no say on hours of operation, whether you have music or not, etc. and especially no interference/disturbance from the tenants of the apartment. They would need to understand they can't just come and hang out like its part of their "space", and that there can be no complaints about noise/activity at too early or too late of an hour. If these things are written into the lease, without conflicting with their existing leases, then if the location is fantastic, I would consider it.
Otherwise I would keep searching. For me, it seems like it might be a source of problems. But whatever they promise, make certain it is in writing, or it's meaningless. Do what's best for you, and think of all the consequences first.
Hi, John... Thanks for your input. I reread my post and I think I missed something. What I meant to say was that there is a building for sale on a main street in the next town over. I would own the building. The rent/lease coming in is $3200/month, plus the tenants cover water, taxes, sewer. The guy leasing is a karate school and I saw a ton of parents in and out of the building at different points in the day. The karate school is in year one of a five year lease. The town is going through a redevelopment and they have nice, wide pedestrian friendly sidewalks. The previous owner was a builder who unexpectedly passed away and the estate is just trying to sell it. It is in amazing condition.