I have not stayed at the Silver City Resort so I can't comment on the actual facility, only the location. It is a little-visited area of the park, so it would definitely be quiet and peaceful there if that is what you are seeking. It is ideally located if you were planning to hike in the Mineral King area of the park, which is mostly visited by hikers. Very few tourists visit the Mineral King area, which is a relatively recent addition to the park. Sequoia National Park was established in 1890 but the Mineral King area did not become part of the park until 1978. If you are planning to spend time hiking in the Mineral King area, or if you just want a quiet getaway, then the Silver City Resort would be appropriate for that.
But if your plan is to tour the highlights of Sequoia and Kings Canyon National Parks, then it may be too far out of the way for that purpose. You would have at least a 1-hour drive, probably more, just to get to the Giant Forest area of Sequoia National Park, then another hour to get to the Grant Grove area of Kings Canyon National Park, then another hour to get down to the Cedar Grove area of Kings Canyon. So staying at Silver City will add at least 2 hours of driving time to each day that you visit those areas of the park. In addition, Highway 198 that goes up into the park is a very winding, curving road; I think that there are over 100 curves between the Ash Mountain entrance and the Giant Forest area of the park. If you are prone to motion sickness I would not want to use that road and would use Highway 180 and the Big Stump entrance to Kings Canyon National Park instead.
The Grant Grove Cabins are located in a popular part of the parks, so there are a lot of people and cars around, but it's still a beautiful location, and it's pretty central, if you are exploring both Sequoia and Kings Canyon National Parks. The Grant Grove is very close by, even walkable to, from the Grant Grove Cabins, and Panoramic Point is not far away, though you would have to drive up to it. The drive from Grant Grove to the Giant Forest area of Sequoia is about 1 hour, and the drive from Grant Grove down into Kings Canyon is also about a 1-hour drive, so the total amount of driving for your trip will be a little less. I don't know if that will make much of a difference, but if you are new to visiting American national parks and new to mountain driving, maybe it will make a difference.
The decision is up to you, of course, but if you want to see a lot of sights all over the parks, the Grant Grove is a better location to use as a base. If instead you wanted a quiet, peaceful getaway without many people and cars around, the Silver City would be a better choice for that.