{"id":38368,"date":"2020-02-20T00:00:31","date_gmt":"2020-02-19T16:00:31","guid":{"rendered":"https:\/\/case.ntu.edu.tw\/blog\/?p=38368"},"modified":"2022-01-03T15:32:38","modified_gmt":"2022-01-03T07:32:38","slug":"%e6%8e%a7%e5%88%b6%e5%a4%a7%e8%b1%a1%e7%9a%84%e5%b0%8f%e8%80%81%e9%bc%a0","status":"publish","type":"post","link":"https:\/\/case.ntu.edu.tw\/blog\/?p=38368","title":{"rendered":"\u63a7\u5236\u5927\u8c61\u7684\u5c0f\u8001\u9f20"},"content":{"rendered":"
\n
\"\u63a7\u5236\u5927\u8c61\u7684\u5c0f\u8001\u9f20\"<\/div>\n<\/div>\n
\n
\n

\u64b0\u6587\uff0f\u9673\u5101\u7ff0<\/strong><\/p>\n

\u5c6c\u6027\u6a21\u578b<\/strong><\/p>\n

\u9019\u662f\u4e00\u7a2e\u96a8\u63d2\u5373\u7528\u7684\u6846\u67b6\uff0c\u53eb\u505aPlug and Play Language Models\uff08PPLM\uff09\uff0c\u53ef\u4ee5\u8b93\u7814\u7a76\u4eba\u54e1\u5c0d\u6a21\u578b\u751f\u6210\u7684\u6587\u5b57\u6709\u66f4\u5927\u7a0b\u5ea6\u7684\u63a7\u5236\u3002PPLM\u5305\u542b\u4e86\u4e00\u500b\u5c6c\u6027\u6a21\u578b\uff08attribute model\uff09\uff0c\u4e5f\u5c31\u662f\u7576\u4f60\u8f38\u5165\u4e00\u6bb5\u8a71\uff0c\u5b83\u53ef\u4ee5\u544a\u8a34\u4f60\u9019\u6bb5\u8a71\u8ddf\u67d0\u500b\u4e3b\u984c\u4e4b\u9593\u7684\u76f8\u95dc\u6027\uff0c\u6bd4\u5982\uff1a<\/p>\n

p<\/span>(<\/span>\u52d5<\/span><\/span><\/span>\u7269<\/span><\/span><\/span>|<\/span><\/span><\/span>\u6211<\/span><\/span><\/span>\u6709<\/span><\/span><\/span>\u4e00<\/span><\/span><\/span>\u96bb<\/span><\/span><\/span>\u5c0f<\/span><\/span><\/span>\u6bdb<\/span><\/span><\/span>\u9a62<\/span><\/span><\/span>)<\/span>=<\/span>0.2<\/span><\/span><\/span><\/span><\/div>\n
p<\/span>(<\/span>\u79d1<\/span><\/span><\/span>\u5b78<\/span><\/span><\/span>|<\/span><\/span><\/span>\u6211<\/span><\/span><\/span>\u6709<\/span><\/span><\/span>\u4e00<\/span><\/span><\/span>\u96bb<\/span><\/span><\/span>\u5c0f<\/span><\/span><\/span>\u6bdb<\/span><\/span><\/span>\u9a62<\/span><\/span><\/span>)<\/span>=<\/span>0.00001<\/span><\/span><\/span><\/span><\/div>\n
p<\/span>(<\/span>\u653f<\/span><\/span><\/span>\u6cbb<\/span><\/span><\/span>|<\/span><\/span><\/span>\u6211<\/span><\/span><\/span>\u6709<\/span><\/span><\/span>\u4e00<\/span><\/span><\/span>\u96bb<\/span><\/span><\/span>\u5c0f<\/span><\/span><\/span>\u6bdb<\/span><\/span><\/span>\u9a62<\/span><\/span><\/span>)<\/span>=<\/span>0.05<\/span><\/span><\/span><\/span><\/div>\n

\u8aaa\u660e\u300c\u6211\u6709\u4e00\u96bb\u5c0f\u6bdb\u9a62\u300d\u9019\u53e5\u8a71\u6bd4\u8f03\u6709\u53ef\u80fd\u51fa\u73fe\u5728\u8207\u300c\u52d5\u7269\u300d\u9019\u500b\u4e3b\u984c\u76f8\u95dc\u7684\u5c0d\u8a71\u4e2d\uff0c\u53ef\u80fd\u6027\u9ad8\u65bc\u300c\u653f\u6cbb\u300d\u6216\u300c\u79d1\u5b78\u300d\u76f8\u95dc\u7684\u5c0d\u8a71\u3002<\/p>\n

\u689d\u4ef6\u8a9e\u8a00\u6a21\u578b<\/strong><\/p>\n

\u5c07\u5c6c\u6027\u6a21\u578b\u5728PPLM\u6846\u67b6\u4e0b\u8207\u8a9e\u8a00\u6a21\u578b\uff08\u5982GPT-2\uff09\u7d50\u5408\uff0c\u5c31\u8b8a\u6210\u4e86\u4e00\u500b\u689d\u4ef6\u8a9e\u8a00\u6a21\u578b\u3002\u6839\u64da\u8c9d\u6c0f\u5b9a\u7406\uff0c\u6211\u5011\u53ef\u4ee5\u5f97\u5230\u300c\u7279\u5b9a\u4e3b\u984c\u7684\u5c0d\u8a71\u4e2d\u51fa\u73fe\u4e00\u53e5\u5b50\u7684\u6a5f\u7387\u300d\u6b63\u6bd4\u65bc\u300c\u8a72\u53e5\u5b50\u51fa\u73fe\u5728\u7279\u5b9a\u4e3b\u984c\u4e2d\u7684\u6a5f\u7387\u300d\u8207\u300c\u8a72\u53e5\u6a5f\u7387<\/a>\u300d\u7684\u4e58\u7a4d\uff0c\u4e5f\u5c31\u662f\uff1a<\/p>\n

p<\/span>(<\/span>\u53e5<\/span><\/span><\/span>\u5b50<\/span><\/span><\/span>|<\/span><\/span><\/span>\u4e3b<\/span><\/span><\/span>\u984c<\/span><\/span><\/span>)<\/span>\u221d<\/span>p<\/span>(<\/span>\u4e3b<\/span><\/span><\/span>\u984c<\/span><\/span><\/span>|<\/span><\/span><\/span>\u53e5<\/span><\/span><\/span>\u5b50<\/span><\/span><\/span>)<\/span>\u22c5<\/span>p<\/span>(<\/span>\u53e5<\/span><\/span><\/span>\u5b50<\/span><\/span><\/span>)<\/span><\/span><\/span><\/span><\/div>\n

\u5176\u4e2d\u200bp<\/span>(<\/span>\u4e3b<\/span><\/span><\/span>\u984c<\/span><\/span><\/span>|<\/span><\/span><\/span>\u53e5<\/span><\/span><\/span>\u5b50<\/span><\/span><\/span>)<\/span><\/span><\/span><\/span><\/span>\u200b\u53ef\u7531\u5c6c\u6027\u6a21\u578b\u5f97\u5230\uff0c\u200bp<\/span>(<\/span>\u53e5<\/span><\/span><\/span>\u5b50<\/span><\/span><\/span>)<\/span><\/span><\/span><\/span><\/span>\u200b\u548c\u200bp<\/span>(<\/span>\u53e5<\/span><\/span><\/span>\u5b50<\/span><\/span><\/span>|<\/span><\/span><\/span>\u4e3b<\/span><\/span><\/span>\u984c<\/span><\/span><\/span>)<\/span><\/span><\/span><\/span><\/span>\u200b\u5206\u5225\u5c0d\u61c9\u5230\u8a9e\u8a00\u6a21\u578b\u8207\u689d\u4ef6\u8a9e\u8a00\u6a21\u578b\u7684\u8f38\u51fa\u3002<\/p>\n

\u4f7f\u7528PPLM\u7684\u597d\u8655\u5728\u65bc\uff0c\u539f\u6709\u7684\u8a9e\u8a00\u6a21\u578b\u672c\u8eab\u4e26\u4e0d\u9700\u8981\u518d\u6b21\u8a13\u7df4\u6216\u5fae\u8abf\uff0c\u53ea\u8981\u66f4\u65b0\u90e8\u5206\u96b1\u7a7a\u9593\u5373\u53ef\uff0c\u4ee5\u4e0b\u7c21\u55ae\u4ecb\u7d39\u8981\u5982\u4f55\u7d50\u5408\u9019\u5169\u500b\u6a21\u578b\uff08\u4ee5GPT-2\u70ba\u4f8b\uff09\uff1a<\/p>\n

\"\"<\/p>\n

\u6587\u5b57\u8f38\u5165\u70ba\u300c\u9019\u96de\u8089\u5403\u8d77\u4f86\u2026\u2026\u300d\uff0c\u8a9e\u8a00\u6a21\u578b\u539f\u5148\u7684\u8f38\u51fa\u70ba\u300c\u96de\u8089\u5403\u8d77\u4f86\u666e\u666e\u901a\u901a\u300d\u3002\u6211\u5011\u5e0c\u671b\u80fd\u751f\u6210\u66f4\u6b63\u9762\u7684\u53e5\u5b50\uff0c\u65bc\u662f\u900f\u904e\u5c6c\u6027\u6a21\u578b\u6539\u8b8a\u8f38\u51fa\u8a9e\u53e5\u7684\u8a9e\u6c23\uff0c\u4f7f\u5f97\u8a9e\u8a00\u6a21\u578b\u6700\u7d42\u8f38\u51fa\u300c\u96de\u8089\u5403\u8d77\u4f86\u5f88\u7f8e\u5473\u300d\u3002\uff08\u5716\u7247\u4f86\u6e90\uff1aDathathri et al 2019.\uff09<\/p>\n<\/div>\n

    \n
  1. \u6587\u5b57\u8f38\u5165GPT-2\u6a21\u578b\uff0c\u6240\u7522\u751f\u7684\u7d50\u679c\u50b3\u905e\u7d66\u5c6c\u6027\u6a21\u578b\u3002<\/li>\n
  2. \u4ee5\u5c6c\u6027\u6a21\u578b\u53cd\u5411\u50b3\u64ad\u7684\u7d50\u679c\u4f86\u66f4\u65b0GPT-2\u7684\u96b1\u7a7a\u9593\u53c3\u6578\u3002<\/li>\n
  3. GPT-2\u518d\u6b21\u7522\u751f\u65b0\u7684\u8f38\u51fa\u3002<\/li>\n<\/ol>\n

    \u4ee5\u4e0a\u6b65\u9a5f\u53ef\u4ee5\u91cd\u8907\u6578\u6b21\u3002<\/p>\n

    \"\"<\/p>\n

    \u9650\u5236\u8a9e\u8a00\u6a21\u578b\u7684\u66f4\u65b0\u53ea\u5728\u503c\u8f03\u9ad8\u7684\u85cd\u8272\u5340\u57df\u79fb\u52d5\uff0c\u4f7f\u689d\u4ef6\u8a9e\u8a00\u6a21\u578b\u4f9d\u7136\u80fd\u7522\u751f\u901a\u9806\u7684\u53e5\u5b50\u3002(\u5716\u7247\u4f86\u6e90\uff1aDathathri et al 2019)<\/p>\n<\/div>\n

    \u7136\u800c\u6839\u64da\u5c6c\u6027\u6a21\u578b\u53cd\u5411\u50b3\u64ad\u7684\u7d50\u679c\u4f86\u66f4\u65b0GSP-2\u7684\u53c3\u6578\uff0c\u8b8a\u52d5\u5e45\u5ea6\u53ef\u80fd\u904e\u5927\uff0c\u9019\u5c31\u50cf\u5982\u679c\u6211\u5011\u4e00\u5473\u8b93\u6a21\u578b\u7522\u751f\u66f4\u6b63\u5411\u7684\u53e5\u5b50\uff0c\u800c\u4e0d\u52a0\u4e88\u9650\u5236\u7684\u8a71\uff0c\u5f88\u53ef\u80fd\u6703\u7522\u751f\u50cf\u662f\u300c\u5f88\u68d2\uff01\u5f88\u68d2\uff01\u975e\u5e38\u68d2\uff01\u300d\u9019\u985e\u7a7a\u6cdb\u7684\u56de\u61c9\u3002\u4e00\u500b\u597d\u7684\u689d\u4ef6\u8a9e\u8a00\u6a21\u578b\u61c9\u8a72\u4e5f\u8981\u50cf\u4e00\u822c\u7684\u8a9e\u8a00\u6a21\u578b\u4e00\u6a23\uff0c\u751f\u6210\u901a\u9806\u3001\u7b26\u5408\u8a9e\u5883\u7684\u53e5\u5b50\uff0c\u6240\u4ee5\u6211\u5011\u53ef\u4ee5\u900f\u904e\u9650\u5236\u66f4\u65b0\u524d\u8207\u66f4\u65b0\u5f8c\u8a9e\u8a00\u6a21\u578b\u9593\u7684KL\u6563\u5ea6\uff08Kullback-Leibler divergence\uff09\uff0c\u4f7f\u5f97\u5169\u500b\u6a21\u578b\u7684\u5206\u5e03\u662f\u63a5\u8fd1\u7684\uff0c\u5982\u6b64\u4fbf\u80fd\u7522\u751f\u65e2\u901a\u9806\u53c8\u8207\u4e3b\u984c\u76f8\u95dc\u7684\u53e5\u5b50\u3002<\/p>\n

    Uber AI\u5c07PPLM\u6a21\u578b\u8207\u8a66\u7528\u7248\u958b\u653e\u5728\u7db2\u8def\u4e0a<\/a>\uff0c\u8b93\u5927\u5bb6\u90fd\u6709\u6a5f\u6703\u4e00\u8a66\u9019\u500b\u80fd\u64cd\u63a7\u5927\u8c61\u7684\u5c0f\u8001\u9f20\u3002<\/p>\n

     <\/p>\n

    \u53c3\u8003\u8cc7\u6599<\/strong><\/p>\n

      \n
    1. S. Dathathri et al., \u201cPlug and Play Language Models: A Simple Approach to Controlled Text Generation<\/a>\u201c,\u00a0arXiv.org<\/em>, 2019.<\/li>\n
    2. E. Wallace, S. Feng, N. Kandpal, M. Gardner and S. Singh, \u201cUniversal Adversarial Triggers for Attacking and Analyzing NLP<\/a>\u201c,\u00a0arXiv.org<\/em>, 2019.<\/li>\n<\/ol>\n

      (\u672c\u6587\u7531\u6559\u80b2\u90e8\u88dc\u52a9\u300cAI\u5831\u5831\u2500AI\u79d1\u666e\u63a8\u5ee3\u8a08\u756b\u300d\u57f7\u884c\u5718\u968a\u7de8\u8b6f)<\/p>\n<\/div>\n<\/div>\n","protected":false},"excerpt":{"rendered":"

      NLP\u8a9e\u8a00\u6a21\u578b\u5982GPT-2\u6216\u662fBERT\uff0c\u96d6\u7136\u529f\u80fd\u5f37\u5927\u2500\u2500\u4efb\u4f55\u4eba\u90fd\u53ef\u4ee5\u5f9eGithub\u4e0a\u4e0b\u8f09\uff0c\u5feb\u901f\u7522\u751f\u6d41\u66a2\u3001\u4ee5\u5047\u4e82\u771f\u7684\u6587\u5b57\u5167\u5bb9\u2500\u2500\u4f46\u672c\u8cea\u4e0a\u90fd\u5c6c\u65bc\u6587\u672c\u9810\u6e2c\u7cfb\u7d71\uff08\u6839\u64da\u4e0a\u4e0b\u6587\u63a8\u6e2c\u4e0b\u4e00\u500b\u6700\u6709\u53ef\u80fd\u51fa\u73fe\u7684\u5b57\u8a5e\u6216\u53e5\u5b50\uff09\uff0c\u5bb9\u6613\u88ab\u6709\u5fc3\u4eba\u58eb\u64cd\u5f04\uff0c\u7522\u751f\u5e36\u6709\u6b67\u8996\u6216\u6975\u7aef\u8ca0\u9762\u7684\u6587\u5b57\u3002\u5b83\u5011\u5c31\u50cf\u662f\u529b\u91cf\u5f37\u5927\u7684\u5927\u8c61\uff0c\u5982\u679c\u958b\u767c\u8005\u7121\u6cd5\u7d04\u675f\u6a21\u578b\u6240\u751f\u6210\u7684\u6587\u5b57\u5167\u5bb9\uff0c\u80fd\u5be6\u969b\u4ed8\u8af8\u61c9\u7528\u7684\u5730\u65b9\u4fbf\u6703\u8655\u8655\u53d7\u9650\u3002\u4f86\u81eaUber AI\u7684\u7814\u7a76\u54e1\u5c31\u627e\u5230\u4e86\u4e00\u96bb\u53ef\u4ee5\u7528\u4f86\u63a7\u5236\u5927\u8c61\u7684\u5c0f\u8001\u9f20\u3002<\/p>\n","protected":false},"author":21,"featured_media":38369,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[3772,3293],"tags":[5695,5696,5697],"aioseo_notices":[],"jetpack_featured_media_url":"https:\/\/case.ntu.edu.tw\/blog\/wp-content\/uploads\/2022\/01\/PPLM-0.png","_links":{"self":[{"href":"https:\/\/case.ntu.edu.tw\/blog\/index.php?rest_route=\/wp\/v2\/posts\/38368"}],"collection":[{"href":"https:\/\/case.ntu.edu.tw\/blog\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/case.ntu.edu.tw\/blog\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/case.ntu.edu.tw\/blog\/index.php?rest_route=\/wp\/v2\/users\/21"}],"replies":[{"embeddable":true,"href":"https:\/\/case.ntu.edu.tw\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=38368"}],"version-history":[{"count":1,"href":"https:\/\/case.ntu.edu.tw\/blog\/index.php?rest_route=\/wp\/v2\/posts\/38368\/revisions"}],"predecessor-version":[{"id":38370,"href":"https:\/\/case.ntu.edu.tw\/blog\/index.php?rest_route=\/wp\/v2\/posts\/38368\/revisions\/38370"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/case.ntu.edu.tw\/blog\/index.php?rest_route=\/wp\/v2\/media\/38369"}],"wp:attachment":[{"href":"https:\/\/case.ntu.edu.tw\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=38368"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/case.ntu.edu.tw\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=38368"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/case.ntu.edu.tw\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=38368"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}