All Bible translations are in the NRSV unless otherwise noted.
This verse–read literally–would seem to indicate that all mention of idolatrous names are prohibited:
12 Six days you shall do your work, but on the seventh day you shall rest, so that your ox and your donkey may have relief, and your homeborn slave and the resident alien may be refreshed. 13 Be attentive to all that I have said to you. Do not invoke the names of other gods; do not let them be heard on your lips.
14 Three times in the year you shall hold a festival for me.
(Exodus 23:13 emphasis mine)
However, there are a few questions to ask:
1 Are there punishments for this commandment?
2 Did this command apply to foreigners?
3 What does this command mean?
Not All Commandments Have Punishments
The Torah is known for having some harsh punishments but what will surprise some people is that there’s actually quite a lot of commands that have no punishment–specifically I think you can make a case that only some of the negative commands have punishments and none of the positive commands have no punishment. The ones that do appear to have punishments are really just a lack of obtaining the benefit gained from the command.
One example is the Passover command where the Passover–I argue–is a conversion ritual to become a citizen of Israel. This gives you citizenship in Israel which comes with responsibilities and rights. However, if you don’t keep the Passover you are no longer considered a citizen:
I argue that since there is no punishment listed for this commandment and not all commandments have punishments that we can assume there is no punishment for this command.
This Commandment Does Not Apply to Foreigners
It should also come as no surprise that some foreigners are explicitly permitted to break laws without punishments–an example–eating something that dies of itself:
The fat of an animal that died or was torn by wild animals may be put to any other use, but you must not eat it. (Leviticus 7:24)
You shall be people consecrated to me; therefore you shall not eat any meat that is mangled by beasts in the field; you shall throw it to the dogs. (Exodus 22:31)
It’s interesting in one verse it is thought to not be fit for human consumption (something that is only thrown to the dogs) but if there are aliens that disagree with that idea they can still partake in it.
You shall not eat anything that dies of itself; you may give it to aliens residing in your towns for them to eat, or you may sell it to a foreigner. For you are a people holy to the Lord your God. You shall not boil a kid in its mother’s milk. (Deuteronomy 14:21)
I’m not arguing for moral relativism, I think all of God’s commands are good, I just think for practical purposes you couldn’t expect foreigners to follow a lot of the commands when they had not heard of them and it was not the Israelite’s domain of authority to make sure the foreigners obeyed them. Foreigners were given freedom in many regards, however there are some exceptions:
This shall be a statute to you forever: In the seventh month, on the tenth day of the month, you shall deny yourselves, and shall do no work, neither the citizen nor the alien who resides among you. (Leviticus 16:29)
All persons, citizens or aliens, who eat what dies of itself or what has been torn by wild animals, shall wash their clothes, and bathe themselves in water, and be unclean until the evening; then they shall be clean.
These laws give strong indication that not all the laws in the Torah apply to, or should be, enforced on aliens–otherwise why would it need to specify that laws applied to both? There is also some leeway with certain types of commands–for instance I think Lev 17:15 indicates that a foreigner can become unclean just like a native but no one is required to be clean–only with regards to interacting with the temple/tabernacle.
There are places where it says “the same law shall be for a foreigner and a native” however if we take this literally it would contradict with the laws about foreigners versus native servants and the laws about lending at interest to foreigners and natives in the land:
35 If any of your kin fall into difficulty and become dependent on you, you shall support them; they shall live with you as though resident aliens. 36 Do not take interest in advance or otherwise make a profit from them, but fear your God; let them live with you. 37 You shall not lend them your money at interest taken in advance, or provide them food at a profit. 38 I am the Lord your God, who brought you out of the land of Egypt, to give you the land of Canaan, to be your God.
39 If any who are dependent on you become so impoverished that they sell themselves to you, you shall not make them serve as slaves. 40 They shall remain with you as hired or bound laborers. They shall serve with you until the year of the jubilee. 41 Then they and their children with them shall be free from your authority; they shall go back to their own family and return to their ancestral property. 42 For they are my servants, whom I brought out of the land of Egypt; they shall not be sold as slaves are sold. 43 You shall not rule over them with harshness, but shall fear your God. 44 As for the male and female slaves whom you may have, it is from the nations around you that you may acquire male and female slaves. 45 You may also acquire them from among the aliens residing with you, and from their families that are with you, who have been born in your land; and they may be your property. 46 You may keep them as a possession for your children after you, for them to inherit as property. These you may treat as slaves, but as for your fellow Israelites, no one shall rule over the other with harshness.
(Leviticus 25:35-46 emphasis mine) 
In addition “the same law shall be for a foreigner and a native” is only said in the context of certain laws:
48 If an alien who resides with you wants to celebrate the passover to the Lord, all his males shall be circumcised; then he may draw near to celebrate it; he shall be regarded as a native of the land. But no uncircumcised person shall eat of it; 49 there shall be one law for the native and for the alien who resides among you.(Exodus 12:48-49)
21 One who kills an animal shall make restitution for it; but one who kills a human being shall be put to death. 22 You shall have one law for the alien and for the citizen: for I am the Lord your God. 23 Moses spoke thus to the people of Israel; and they took the blasphemer outside the camp, and stoned him to death. The people of Israel did as the Lord had commanded Moses.
28 And the priest shall make atonement before the Lord for the one who commits an error, when it is unintentional, to make atonement for the person, who then shall be forgiven. 29 For both the native among the Israelites and the alien residing among them—you shall have the same law for anyone who acts in error. 30 But whoever acts high-handedly, whether a native or an alien, affronts the Lord, and shall be cut off from among the people.
14 An alien who lives with you, or who takes up permanent residence among you, and wishes to offer an offering by fire, a pleasing odor to the Lord, shall do as you do. 15 As for the assembly, there shall be for both you and the resident alien a single statute, a perpetual statute throughout your generations; you and the alien shall be alike before the Lord. 16 You and the alien who resides with you shall have the same law and the same ordinance.
13 But anyone who is clean and is not on a journey, and yet refrains from keeping the passover, shall be cut off from the people for not presenting the Lord’s offering at its appointed time; such a one shall bear the consequences for the sin. 14 Any alien residing among you who wishes to keep the passover to the Lord shall do so according to the statute of the passover and according to its regulation; you shall have one statute for both the resident alien and the native.
15 On the day the tabernacle was set up, the cloud covered the tabernacle, the tent of the covenant; and from evening until morning it was over the tabernacle, having the appearance of fire.
Exodus 23:13 Means to Not Worship Other Gods
First lets agree on one thing, Exodus 23:13 cannot be taken as it is literally read:
Be attentive to all that I have said to you. Do not invoke the names of other gods; do not let them be heard on your lips. (Exodus 23:13)
There are multiple places where the titles of other gods are mentioned in the Torah such as “Baal” Jdg 2:11, Jdg 2:13, Jdg 3:7 etc . . . https://www.blueletterbible.org/lang/lexicon/lexicon.cfm?Strongs=H1168&t=KJV and “Baal” was indeed the name associated with that god because the Biblical writers replaced names that had “baal” in it with “bosheth” to signal a rejection of “baal.” 
So what does this mean?
Do not “invoke” the names
Hebrew is a language more focused on action than english. “To invoke” in English when associated with a god has the idea of “invoking to worship.” “Invoke” is a better translation of the word than the KJV has (“remember”) but even “to invoke” doesn’t give the idea that this word seems to be associated with action almost all the time: https://www.blueletterbible.org/lang/lexicon/lexicon.cfm?Strongs=H2142&t=KJV
the “names” of other gods
https://www.blueletterbible.org/lang/lexicon/lexicon.cfm?page=3&strongs=H8034&t=KJV#lexResults If we use the “name” of God for a reference point–it is actions that God does that reflect his authority or character:
5 The Lord descended in the cloud and stood with him there, and proclaimed the name, “The Lord.” 6 The Lord passed before him, and proclaimed,
“The Lord, the Lord,
a God merciful and gracious,
slow to anger,
and abounding in steadfast love and faithfulness,
7 keeping steadfast love for the thousandth generation,
forgiving iniquity and transgression and sin,
yet by no means clearing the guilty,
but visiting the iniquity of the parents
upon the children
and the children’s children,
to the third and the fourth generation.”
8 And Moses quickly bowed his head toward the earth, and worshiped.
do not let them be “heard“
Even “heard” can mean “heed” or “pay attention” with your actions:
But they did not listen or incline their ear, to turn from their wickedness and make no offerings to other gods. (Jer 44:5)
17 And to the man he said,
“Because you have listened to the voice of your wife,
and have eaten of the tree
about which I commanded you,
‘You shall not eat of it,’
cursed is the ground because of you;
in toil you shall eat of it all the days of your life;
and Sarai said to Abram, “You see that the Lord has prevented me from bearing children; go in to my slave-girl; it may be that I shall obtain children by her.” And Abram listened to the voice of Sarai. (Gen 16:2)
And the angel of the Lord said to her,
“Now you have conceived and shall bear a son;
you shall call him Ishmael,
for the Lord has given heed to your affliction.
I would argue that none of this is about words or names but about actions and possibly worship. At most this restriction on speech would prevent you from “invoking” the name of another god in a worshipful way but it wouldn’t prevent you from talking about that god, the worship of that god, or possibly even advocating for such worship. There are other verses that imply that the “name on the lips” and “name being remembered” are indeed idioms for worship:
Be attentive to all that I have said to you. Do not invoke h2142 the names h8034 of other gods; do not let them be heard on your lips. h6310 (Exodus 23:13)
For I will remove the names of the Baals from her mouth, h6310 and they shall be mentioned h2142 by name h8034 no more. (Hosea 2:17)
On that day, says the Lord of hosts, I will cut off the names h8034 of the idols from the land, so that they shall be remembered h2142 no more; and also I will remove from the land the prophets and the unclean spirit. (Zechariah 13:2)
Another reason we can see this active worship interpretation is correct is because of the clear punishment for idolatry elsewhere and the fact that it only prohibits people advocating the worship of other gods “in secret” not openly:
6 If anyone secretly entices you—even if it is your brother, your father’s son or your mother’s son, or your own son or daughter, or the wife you embrace, or your most intimate friend—saying, “Let us go worship other gods,” whom neither you nor your ancestors have known, 7 any of the gods of the peoples that are around you, whether near you or far away from you, from one end of the earth to the other, 8 you must not yield to or heed any such persons. Show them no pity or compassion and do not shield them. 9 But you shall surely kill them; your own hand shall be first against them to execute them, and afterwards the hand of all the people. 10 Stone them to death for trying to turn you away from the Lord your God, who brought you out of the land of Egypt, out of the house of slavery. 11 Then all Israel shall hear and be afraid, and never again do any such wickedness.
12 If you hear it said about one of the towns that the Lord your God is giving you to live in, 13 that scoundrels from among you have gone out and led the inhabitants of the town astray, saying, “Let us go and worship other gods,” whom you have not known, 14 then you shall inquire and make a thorough investigation. If the charge is established that such an abhorrent thing has been done among you, 15 you shall put the inhabitants of that town to the sword, utterly destroying it and everything in it—even putting its livestock to the sword. 16 All of its spoil you shall gather into its public square; then burn the town and all its spoil with fire, as a whole burnt offering to the Lord your God. It shall remain a perpetual ruin, never to be rebuilt. 17 Do not let anything devoted to destruction stick to your hand, so that the Lord may turn from his fierce anger and show you compassion, and in his compassion multiply you, as he swore to your ancestors, 18 if you obey the voice of the Lord your God by keeping all his commandments that I am commanding you today, doing what is right in the sight of the Lord your God.
(Deuteronomy 13:6-18 emphasis mine)
2 If there is found among you, in one of your towns that the Lord your God is giving you, a man or woman who does what is evil in the sight of the Lord your God, and transgresses his covenant 3 by going to serve other gods and worshiping them—whether the sun or the moon or any of the host of heaven, which I have forbidden— 4 and if it is reported to you or you hear of it, and you make a thorough inquiry, and the charge is proved true that such an abhorrent thing has occurred in Israel, 5 then you shall bring out to your gates that man or that woman who has committed this crime and you shall stone the man or woman to death. 6 On the evidence of two or three witnesses the death sentence shall be executed; a person must not be put to death on the evidence of only one witness. 7 The hands of the witnesses shall be the first raised against the person to execute the death penalty, and afterward the hands of all the people. So you shall purge the evil from your midst.
This command does not prohibit speech except possibly the worship of a god by invoking his name. However, advocating for worship or speaking about the god may not have been prohibited. Even if it was this would only apply to natives not foreigners. In addition advocating for worship elsewhere is only prohibited if it is done in secret and not allowed to be evaluated by the community.
1 Are there punishments for this commandment? No
2 Did this command apply foreigners? No.
3 What does this command mean? Not to worship other gods and possibly not advocate for such.
 Just to clarify, I don’t think the word translated “advance” interest is talking about something different than regular interest, see the following:
Two Types of Interest?
The Hebrew term for interest, נֶשֶׁךְ (nešekh), appears in all three of the Torah’s legal collections prohibiting interest, and in Leviticus, it is paired with תַרְבִּית (see also Ezek 18:8, 13; Prov 28:8), which also means interest. This seeming redundancy has led many commentators to understand these terms as two different types of interest.
* NJPS translates נֶשֶׁךְ as “advanced interest” (i.e., paid at the time of the loan or earlier) and תַרְבִּית as “accrued interest” (i.e., paid upon the repayment of the loan or after).https://www.thetorah.com/article/what-is-wrong-with-charging-interest (accessed 2021-02-06)
* The Mishnah (Bava Metzia 5:1), however, understands נֶשֶׁךְ as an advance agreement that the borrower will pay back more than the amount of the original loan and תַרְבִּית as paying back the loan of one form of produce with another form of produce that has risen in value, that the borrower did not have on hand at the time of the deal.
Although the verse in Leviticus may very well be including more than one form of interest, it is unlikely that the inverse is the case, namely that Exodus and Deuteronomy meant to forbid only one form of interest. Instead, these sources probably meant נֶשֶׁךְ as a catch-all term for interest. In all three sources, the point appears to be that an Israelite should not apply the “bite” of interest to loans given to his poor brethren.
 Ishbosheth, also spelled Isboseth, also called Ishbaal, or Eshbaal, (flourished 11th century BC), in the Old Testament (II Samuel 2:8–4:12), fourth son of King Saul and the last representative of his family to be king over Israel (the northern kingdom, as opposed to the southern kingdom of Judah). His name was originally Ishbaal (Eshbaal; I Chronicles 8:33; 9:39), meaning “man of Baal.” Baal, which could mean “master,” was a title of dignity. Because the name came to be increasingly associated with Canaanite fertility gods, Hebrew editors later substituted bosheth, meaning “shame,” for baal.https://www.britannica.com/biography/Ishbosheth (accessed 2021-02-07)