Silver nanoparticles (AgNPs) are widely used in household products and in medicine due to their antibacterial and to wound healing properties. In recent years, there is also an effort for their use in biomedical imaging and photothermal therapy. The primary reason behind the effort for their utility in biomedicine and therapy is their unique plasmonic properties and easy surface chemistry for a variety of functionalizations. In this study, AgNPs modified with glucose, lactose, oligonucleotides and combinations of these ligands are investigated for their cytotoxicity and cellular uptake in living non-cancer (L929) and cancer (A549) cells. It is found that the chemical nature of the ligand strongly influences the toxicity and cellular uptake into the model cells. While the lactose-and glucose-modified AgNPs enter the L929 cells at about the same rate, a significant increase in the rate of lactose-modified AgNPs into the A549 cells is observed. The binding of oligonucleotides along with the carbohydrate on the AgNP surfaces influences the differential uptake rate pattern into the cells. The cytotoxicity study with the modified AgNPs reveals that only naked AgNPs influence the viability of the A549 cells. The findings of this study may provide the key to developing effective applications in medicine such as cancer therapy.