I wouldn't unless the lease terms are very favorable. i.e. Landlord has no say on hours of operation, whether you have music or not, etc. and especially no interference/disturbance from the tenants of the apartment. They would need to understand they can't just come and hang out like its part of their "space", and that there can be no complaints about noise/activity at too early or too late of an hour. If these things are written into the lease, without conflicting with their existing leases, then if the location is fantastic, I would consider it.
Otherwise I would keep searching. For me, it seems like it might be a source of problems. But whatever they promise, make certain it is in writing, or it's meaningless. Do what's best for you, and think of all the consequences first.
my 2 cents.